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FOREWARD 


The  Walter  Reed  Army  Institute  of  Research  (WRAIR),  the  host  for  the 
Seventeenth  Conference  on  the  Design  of  Experiments  In  Army  Research,  Development 
and  Testing,  has  as  its  basic  mission  to  provide  the  medical  research  and 
professional  graduate  training  required  by  the  Army  to  fulfill  its  role  in. 
National  Defense,  Since  strtistics  and  ocher  scientific  disciplines  play  an 
ever  Increasing  role  in  medical  fields,  it  is  not  surprising  that  Colonel 
Edward  L,  Buescher,  Director  and  Commandant  of  WRAIR,  was  pleased  to  have  his 
installation  serve  as  host  to  this  design  of  experiments  conference.  Colonel 
Hinton  J.  Baker  wee  asked  to  act  as  Chairman  on  Local  Arrangements.  He  was 
assisted  in  this  capacity  by  Douglas  Tang,  Those  in  attendance  are  in  debt 
to  these  two  gentlemen  for  so  abling  handling  the  many  housing  and  transportation 
problems  that  arose  during  the  course  of  the  meeting.  Major  General  Colin  F, 
Vorder  Bruegge,  Commanding  General  of  the  Walter  Reed  Army  Medical  Center,  in 
his  welcoming  remarks,  as  well  as  in  his  comments  at  the  banquet,  made  the 
audience  feel  that  their  scientific  accomplishments  were  helping  medical  research 
to  develop  in  many  areas. 

This  is  the  second  time  that  WRAIR  has  served  as  host  to  one  of  these 
conferences.  The  eighth  conference  in  this  series,  sponsored  by  the  Army 
Mathematics  Steering  Committee  on  behalf  of  the  Office  of  the  Chief  of  Research 
and'  Development,  Department  of  the  Army,  was  held  24-26  October  1962  at  the 
Walter  Reed  Army  Medical  Center,  Of  the  180  attendees  at  the  eighth  conference, 

21  attended  the  seventeenth  conference.  Dr.  Stefano  Vlvona,  now  with  the  American 
Cancer  Society,  was  the  local  chairman  for  the  earlier  conference.  He  was  one 
of  the  repeat  attendees.  Dr.  Herbert  C.  Batson  was  one  of  the  invited  speakers 
at  the  1962  conference.  At  the  banquet  of  this  meeting  Dr.  Baker  presented  him 
a  citation  for  his  outstanding  scientific  contribution  to  the  field  of  medicine. 
Drs,  Marvin  Zelen  and  George  Lavin  presented  papers  at  both  conferences.  At 
the  eighth  Conference  Zelen  contributed  a  paper  which  was  jointly  authored  by 
Dr,  Bsdrlg  Kurkjlan  and  himself.  At  the  1971  conference  he  was  an  invited 
speaker.  Another  invited  speaker  who  attended  both  meetings  was  Professor 
Bernard  Greenberg,  It  is  Interesting  to  note  that,  among  the  others  who 
attended  both  meetings  and  participated  in  both  programs  as  chairman,  panelist, 
or  by  contributing  p  tpere  were  0.  P,  Bruno,  A,  C.  Cohen,  Francis  DrcaBel,  Henry 
Ellner,  Walter  D,  Foster,  Frank  E.  Grubbs,  Boyd  Harshbarger,  Bsdrlg  Kurkjlan, 
Clifford  J,  Maloney  and  Beatrice  S,  Orleans, 

We  note'  a  few  more  statistics  about  the  two  meetings.  The  earlier  one  had 
forty-four  more  registered  persons.  One  of  the  features  of  these  meetings,  the 
clinical  session,  has  gained  in  popularity.  In  the  1962  conference  there  were 
two  of  these  sessiona  with  only  three  papers,  while  in  1971  there  were  five 
sessions  In  which  eleven  clinluL— -type  papers  were  presented.  The  percentage 
increase  In  the  papers  for  the  technical  sessions  were  not  as  great,  still 
there  wee  some  Increase,  namely  twenty-six  contributed  papers  in  1962  as  compared 
with  thirty-two  technical  papers  presented  this  year. 

One  of  the  requirements  the  chairman  makes  of  the  members  of  his  Program 
Committee  is  that  they  name  at  least  one  speaker  whose  topic  will  be  of  special 
interest  to  the  members  of  the  host  Installation.  This  year  that  request  wae 
taken  seriously.  Most  of  the  invited  speakers  touched  on  topics  that  were  of 
interest  to  those  in  the  medical  field.  This  can  be  seen  from  the  following 
list  of  Invited  speakers  and  the  titles  of  their  addresses t 

"The  Role  of  Mathematical  Sciences  in  Biomedical  Research"  by 

Professor  Marvin  Zelen 


iii 


Preceding  pigs  blink 


"Randomized  Response:  A  New  Survey  Tool  Co  Collect  Data  of  a 
Personal  Nature"  by  Professor  Bernard  G.  Greenberg 

"Classification  and  Clustering  Techniques  in  Data  Analysis" 
by  Dr,  Geoffrey  H,  Ball 

"Hotelling's  Weighing  Desings" 
by  Professor  K.  S,  Saner jee 

"The  Comparison  of  Proportions:  A  review  of  Significance  Tests, 

Confidence  Intervals  and  Adjustments  for  Stratification" 
by  Dr.  John  J.  Gart 

Let  us  take  this  opportunity  to  thank  members  of  the  program  committee 
(David  Ailing,  Hinton  J.  Baker,  Francis  Dressel,  Walter  Foster,  Fred  Frlshman, 
Bernard  Greenberg,  Bernard  Harris,  Boyd  Harshbarger,  Allyn  Kimball,  Clifford  J. 
Maloney,  Herbert  Solomon  and  Douglas  Tang)  for  their  recommendations  for  invited 
speakers,  as  well  as  their  active  part  In  the  conduction  of  this  meeting.  We 
would  be  remiss  in  our  duties  if  we  did  not  give  due  praise  to  all  those  indi¬ 
viduals  who  gave  contributed  papers.  For  truly,  without  their  help,  this  meeting 
could  not  have  been  a  successful  scientific  conference. 

Members  of  the  Army  Mathematics  Steering  Committee  have  asked  that  the 
proceedinga  of  this  conference  be  made  available  to  those  in  attendance  for 
further  study  of  the  contents  of  the  presented  papers,  and  to  those  interested 
in  the  copies  covered  at  the  conference  but  who  were  unable  to  attend, 


Francis  G,  Dressel 
Score tary 


Frank  E.  Grubbs 
Conference  Chairman 
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RANDOMIZED  RESPONSE!  A  NEW  SURVEY  TOOL 
TO  COLLECT  DATA  OP  A  PERSONAL  NATURE 

Bernard  G.  Greenberg 
Janes  S.  Abernathy 

Department  of  Blostntlstics ,  University  of  North  Carolina  at  Chapel  Hill 

Daniel  G.'  Horvltz 

Statistics  Research  Division,  Research  Triangle  Institute, 

Research  Triangle  Park,  North  Carolina 

INTRODUCTION 

Refusal  to  respond  and  an  untruthful  answer  from  a  respondent  are  recognized 
as  two  principal  sources  of  nonsampling  error  that  can  Influence  sample  estimates 
involving  surveys  of  human  populations.  These  potential  Bourccs  of  bias  are  more 
prevalent  in  surveys  which  involve  sensitive  or  embarrassing  questions.  Warner^#':1'' 
the  developer  of  the  randomized  response  technique,  designed  the  procedure  in 
order  to  reduce  or  eliminate  bias  introduced  when  respondents  In  surveys  dellb- 

a 

eratcly  give  false  Information  or  refuse  to  answer  questions  of  a  personal  or 
stigmatizing  nature.. 

Randomized  response  is  a  relatively  new  and  exciting  statistical  technique 
of  great  potential  In  surveys  of  human  populations.  Although  the  method  ie  still 
in  the  infancy  stage,  much  growth  and  development  have  occurred  since  its  crea¬ 
tion  in  1965. 

HISTORICAL  DEVELOPMENTS 

Qualitative  response.  In  his  original  paper,  Warner  considered  the  case 
where  a  proportion  it  of  the  population  (say  Group  A)  possessed  some  sensitive 
characteristic  whi  le  the  remainder  of  the  population  did  not  possess  the 
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characteristic.  The  objective  was  to  estimate  ir.  With  the  aid  of  a  randomizing 
device,  the  respondent  selects  one  of  the  following  statements  by  chance, 


1  am  a  member  of  Group  A.  [with  a  probability  of  P] 

1  am  not  a  member  of  Group  A.  [with  ..  rrob ability  of  (1-P)] 

... 

and  answers  "Yes"  or  "No"  to  whichever  one  of  the  two  statements  is  selected. 

The  interviewer  does  not  know  which  statement  was  actiially  selected  through 

.  ! 

the  randomization  process  and,  therefore,  does  not  know  to  which  statement  the 

■1  ' 

respondent's  reply  refers.  Only  the  respondent  knows  to  which  one  of  the  two 
statements  his  reply  io  addressed. 

I' 

The  rationale  underlying  tho  randomized  response  procedure  is  th«t,  since 

the  respondent  can  answer  a  sensitive  question  without  revealing  his  personal 

.  •stfr  W 

situation,  potential  stigma  and  embarrassment  on  the  part  of  the  respondent  are 

removed.  Under  these  conditions  there  is  no  longer  any  need  to  refuse  to 
respond  or  to  give  an  incorrect  or  evasive  answer.  If  respondents  are  con¬ 
vinced  that  the  procedure  guarantees  anonymity  in  this  respect,  it  follows  chat 
cooperation  and  validity  of  response  should  be  improved. 

The  randomized  response  procedure  dearly  does  not  permit  interpretation 
of  individual  responses  and,  in  fact,  is  not  designed  for  that  purpose.  Its 
sole  purpose  is  to  determine  IT  in  a  group  of  respondents.  For  analytical  pur-j*r. 
poses  the  total  sample  may  be  subdivided  according  to  age,  race,  and  other 
available  characteristics  in  order  to  study  the  relationship  of  the  various 
values  of  it  in  the  subgroups. 

Warner  showed  that  the  maximum  likelihood  estimate  of  it  is  unbiased,  If 


persons  are  encouraged  by  the  technique  to  tell  the  truth,  and  its  value  is 
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where 

n  Is  the  sample  size  of  which  m  persons  reply  in  the  affirmative. 

Jhe  variance  of  the  estimate,  as  given  by  Warner,  is 

Vbv(ti)  -  S-iiSi  +  likli- 
.  n  n(2P-X)* 

[2] 

Abul-Eln  et_  all  1  extended  Warner’s  model  to  the  trichotomoua  case  designed 
to  estimate  the  proportions  of  three  related,  mutually  exclusive  groups,  one  or 
two  of  which  possessed  a  sensitive  characteristic.  The  model  was  further 
extended  to  estimate  any  j  proportions  (j>3)  when  all  the  j  group  characteristics 
are  mutually  exclusive,  with  at  least  one  and  at  most  j-l  of  then  sensitive.  The 
reason  behind  this  extension  was  to  provide  theory  for  the  multichotomaus  situation 
which  is  often  found  in  practice.  The  solution  to  this  problem  was  found  to  lie 
in  choosing  a  new,  nonoverlapping  sample  with  a  different  value  of  P  for  e.very 
additional  parameter  to  be  estimated, 

[31 

Following  a  suggestion  by  Simmons,  Abul-Ela  investigated  and  describod  a 
variation  of  the  Warner  technique  known  as  the  unrelated  question  model.  As 
indicated  above,  the  Warner  technique  concerned  two  questions  (or  statements) 
which  are  inversely  related  to  the  sensitive  characteristic.  The  unrelated  model 
is  predicated  on  the  assumption  that  confidence  in  the  anonymity  of  the  technique 
would  be  increased  if  two  unrelated  questions  were  used,  one  pertaining  to  the 
sensitive  characteristic  and  the  other  to  a  nonsonsitive,  innocuous  condition. 
Horvitz,  et_  _al^  further  discussed  this  modification  and  presented  results  from 
two  field  studies.  Greenberg  et_  alj'" ^  studied  the  theoretical  framework  of  the 
unrelated  question  model  with  respect  to  estimation  of  the  sensitive  attribute, 
variance  of  the  estimates,  effect  of  untruthful  reporting,  selection  of  the 
unrelated  characteristic,  and  other  design  properties.  Mean  square  error  efficiency 
of  the  unrelated  question  model  versus  the  original  Warner  model  was  reported  to 
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favor  the  unrelated  question  model.  Further  reduction  in  mean  square  error  was 
shown  to  accrue  when  the  frequency  of  the  unrelated  neutral  characteristic  was 
known  in  advance.  Furthermore,  a  method  of  formulating  the  unrelated  question 
which  insures  a  priori  knowledge  of  its  parameters  was  described.  This  latter 
suggestion  was  made  to  the  senior  author  by  Richard  Morton  of  the  University  of 
Sheffield  and  it  Was  field-tested  by  Greenberg  £t^  al^?  with  favorable  results. 
The  results  of  this  application  are  shown  in  the  next  section  in  an  example  where 
data  on  emotional  problems  are  described. 

The  maximum  likelihood  estimate  of  the  sensitive  attribute,  let  us  call  it 


now,  In  the  unrelated  question  model  (assuming  that  the  proportion  of  the 


population  possessing  the  unrelated  characteristic,  ir^,  is  known  in  advance)  and 
its  variance  are 

X-Ttv(l-1*) 


- 


( 


V«r(tJirY)  - 


nP 


where 

X  *>  probability  that  a  "Yes"  answer  will  be  reported,  and 

P  =*  probability  that  the  sensitive  question  is  selected  by  the  respondent. 

To  obtain  an  estimate  of  one  simply  substitutes  the  observed  value  of 

X,  say  for  its  expected  value  in  the  above  equation.  £  is  defined  as 
where  n  and  m  have  the  same  definition  as  in  equation  (1). 

Although  highly  desirable,  it  is  not  mandatory  that  an  estimate  of  ir^  be 
known  in  advance.  There  may  be  situations  where  irv  is  not  known  and  not 


amenable  Co  a  priori  estimation.  Under  these  conditions  the  sample  design 
can  be  «lt»yed  to  permit  estimation  not  only  of  the  sensitive  characteristic 
by  ir^  as  well.  This  may  be  accomplished  by  selecting  two  independent  samples 
of  alecs  and  n2  from  the  population.  There  are  two  devices  end  they  would 
be  arranged  in  such  a  way  that  tha  probabilities  of  selecting  the  sensitive 
question  in  the  two  subsamplcs  (P^  and  ?2,  respectively)  are  different.  The 

. . .  A  . . 1  ' 

resulting  proportions  of  "Yes"  responses  in  the  two  subsamples,  X^  end  Ag*  would 
also  be  different.  Estimation  formulae,  assuming  fly  not  known  in  advance,  are 


with  variances, 


.\1(i-p2)-x2(i-p1> 


prp2 


*ip2"Vi 

VP1 


V<V 


1  pi 

(PrP2)2 


a-xpd-Pj)1  Xjd-Apa-p^* 

- - -  + - 


V<0y) 


<VP1>‘ 


>l(l-X1)P*  X2(l-X2)pJ 
nl  n2 


Gould  e£  al  ^  considered  the  unrelated  question  method  with  two  trials 
per  respondent  in  an  attempt  to  develop  models  of  respondent  behavior  and 
apply  these  models  to  survey  results  on  illegitimate  births  in  which  randomized 
response  was  used. 


Barksdale^  extended  previous  work  In  randomized  response  to  permit  the 
estimation  of  the  coefficient  of  correlation  of  two  related  traits,  st  least 
one  of  which  is  stigmatizing.  The  potential  value  of  estimating  the  proportion 
of  individuals  with  two  attributes  was  also  treated. 

Quantitative  response.  Most  of  the  randomized  response  work  to  date 
has  buen  concerned  with  refining  the  technique  for  use  in  personal  interviews 
using  questions  of  &  qualitative  nature  requiring  only  a  'Tea"  or  "No"  response. 
The  technique  need  not  be  restricted  to  nominal  scale  data,  however.  It  hae  wide 
application  in  the  area  of  quantitative  response  and  study  is  being  directed 
toward  further  development  of  the  method  in  this  area.  Cruenberg  ejt  al^  have 
presented  theory  underlying  the  quantitative  application  of  the  randomized 
response  procedure  including  unbiased  estimates  of  the  population  mean  and 
variance  of  both  the  sensitive  end  non-sensitive  distributions. 

The  quantitative  model  differs  from  the  qualitative  in  that  the  questions 
are  designed  to  elicit  a  response  in  quantitative  terms  rather  than  "Yes"  or 
"No".  The  responses  to  both  the  sensitive  and  non-sensitive  question  must  be  of 
the  same  relative  magnitude,  thus  making  it  impossible  to  identify  from  the 
response  which  question  has  been  answered.  The  overall  distribution  of  responses 
is  comprised  of  numerical  answers  to  both  questions.  This  distribution  must  be 
separated  in  order  to  provide  estimates  of  the  parameters  of  interest,  viz.,  the 
population  mean  of  both  the  sensitive  and  non-sensitive  distribution,  and  their 
variance*. 

Assuming  two  independent,  non-overlapping  samples  of  sizes  and  n,,, 

unbiased  estimators  for  the  mean  of  the  sensitive  and  non-sensitive  distributions, 

(81 

M^(*nd  respectively,  are  shown1  1  to  be 
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(3) 


..  Si*, 

Aw:- 

h-v 


X: 

B 

I 


<1-P2)Z1  -  (l-P^gg 


prp2 


»*» 


?A  -  *ili 

Vpi 


where 

P.  *  probability  thst  the  sensitive  question  la  selected  by  the 

1  .  . ' . Ir .  <-■  . . ' 

respondent  In  sample  i(i-l,7.) , 

\  "  mean  response  of  sample  1(1*1 ,2) 


4  A 

The  variances  of  the  two  estimators,  y^  and  y^  were  found  to  be 


V(y  )  - 1 — .  m-v 


<W 


7  lCi-P2>,v<ft>  +  Ci-P1>*vcz2)] 


V<n  >  .  — -1—-  IP*V(2  )  +  P*V(Z,)] 

<VP1> 


where 


(4) 


v(5l>  ■  W1). 

The  variances  of  the  estimators  can  be  conveniently  estimated  by  using 
the  sample  variances  s£  in  (4):  9(5^)  *  and  0(f2)  “  ®2^n2* 

Other  developments.  Since  msny  surveys  are  self-administered  by  means 
of  mail  questionnaires  or  other  media,  extension  of  randomised  fcaponse  to 
auch  sources  of  information  Is  essential.  The  North  Carolina  group  of 
investigators  in  currently  exploring  the  development  of  a  randomisation 
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device  which  can  be  implemented  in  the  questionnaire.  This  technique  presents 
no  substantive  problems  in  statistical  theory  different  from  those  already 
studied  but  does  present  problems  in  question  design*  format*  respondent 
acceptance*  and  illiteracy. 

FIELD  APPLICATIONS 

The  first  field  application  of  the  unrelated  question  technique  was  con- 

rzi 

ducted  by  the  Research  Triangle  Institute  in  October,  1965,  and  involved 
personal  Interviews  in  a  total  sample  of  104  white  and  44  black  households  in 
which  it  was  known  that  a  live  birth  had  occurred  within  the  previous  two 
sionths.  It  was  also  known  that  18.9  per  cent  of  the  births  were  illegitimate 
according  to  the  birth  certificate  on  file  with  the  North  Carolina  State  Board 
of  Health.  The  respondents  were  asked  to  select  a  card  from\a  shuffled  dock  of 
50  cards  and  to  answer  "Yes"  or  “No"  to  the  statement  printed  on  the  card.  The 
two  statements  used  in  the  deck  were! 

1.  There  was  a  baby  born  in  this  household  after  January  1, 

1965*  to  an  unmarried  woman  who  was  living  here. 

2.  I  was  bom  in  North  Carolina. 

The  objective  was  to  estimate  the  proportion  of  all  households  with  a 
birth  to  an  unmarried  woman.  The  results  were  in  remarkable  agreement  with 
the  known  proportions  of  illegitimacy  in  each  race  in  the  sample. 

North  Carolina  Abortion  Study.  The  North  Carolina  Abortion  StuAy^®'^ 
was  a  major  field  survey  in  1968  to  test  the  practicability  of  the  randomised 
response  procedure  in  eliciting  information  of  a  still  more  sensitive  nature 
from  the  general  population.  Information  was  sought  on  a  variety  of  subjects 
Including  illegally  induced  abortion,  oral  contraceptive  use,  emotional 
problems,  and  income.  Each  of  theso  areas  of  interest  utilized  an  independent 


sample  of  women  18  years  old  and  over  from  Che  combined  population  of  five 
metropolitan  areas  of  the  state. 

In  deriving  estimates  of  induced  abortion,  the  randomization  device  con¬ 
sisted  of  a  small,  transparent,  sealed,  plastic  box  with  two  questions  printed 
on  the  top,  easily  legible  to  the  respondent} 

1.  I  was  pregnant  at  some  time  during  the  pest  12  months' 
and  had  an  abortion  which  ended  the  pregnancy. 

2.  I  was  born  in  the  month  of  April.' 

The  first  printed  statement  had  a  small,  red  ball  in  front  of  it;  the 
second  statement  had  a  blue  ball  in  the  same  location.  Inside  the  box  there 
weru  35  red  balls  and  15  blue  balls.  The  respondent  was  asked  to  shake  the 
box  of  balls  thoroughly,  and  to  tip  the  box  allowing  one  of  the  freely  moving 
balls  to  appear  in  a  "window"  located  in  the  device  and  clearly  visible  to 
the  respondent.  The  color  of  the  bell  which  appeared  in  the  window  determined 
which  of  the  two  statements  the  respondent  answered  with  a  simple  "Yes"  or 
"Ho".  If  a  red  ball  appeared,  she  answered  the  abortion  statement;  if  a  blue 
ball  appeared,  she  answered  the  born-in-April  statement.  Each  respondent  was 
urged  to  experiment  with  the  plastic  box  several  times  to  assure  herself  that 
both  colors  of  beeds  could  appear  in  the  window.  Color  blindenss  is  no  problem 
with  female  respondents. 

On  the  direction  of  the  interviewer,  the  respondent  then  shook  the  box 
to  answer  the  quention  officially.  The  interviewer  was  some  distance  away 
from  the  respondent  so  that  she  was  unaware  which  statement  had  been  selected. 
The  respondent's  reply  of  "Yea"  or  "No"  was  recorded  by  the  interviewer  with 
no  knowledge  concerning  which  statement  had  been  answered. 

Subsequent  estimates  of  induced  abortion  in  the  population  of  women  18-44 
years  of  age  indicated  that  3.4  per  cent  of  them  had  experienced  an  induced 
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abortion  during  the  previous  year.  By  racial  groups  the  estimates  were  1.4 
per  cent  for  whites  and  6.8  per  cent  for  nonwhites.  There  is  no  way  these 
findings  can  be  absolutely  validated.  They  do  appear  to  be  within  the  realm 
of  reason,  however,  based  upon  limited  evidence  available  for  comparison. 

The  women  who  were  asked  about  abortion  during  the  past  year  were  also 
queried  about  use  of  Che  contraceptive  pill.  The  procedure  used  was  exactly 
the  same  as  that  used  for  abortion,  except  that  the  statements  were  different, 
as  follow^ . 

1.  I  am  now  taking  "the  pill"  to  prevent  prognancy. 

2.  I  was  born  In  the  month  of  April. 

The  results  Indicated  that  one-fourth  of  the  women  were  taking  oral  contra¬ 
ceptives  at  the  time  of  the  survey.  A  higher  estimate  was  found  for  non¬ 
white  women  (28.4%)  than  for  white  women  (23. 1%)  and  thie  concurred  with 
previous  expectations  because  of  family  planning  programa  conducted  In  some 
areas  for  poverty  groups. 

Ir  another  sample  designed  to  obtain  information  on  emotional  problems, 
the  following  set  of  questions  wee  used  with  women  31  years  old  or  over: 

1.  At  some  time  during  my  life  I  had  an  emotional  problem 
which  caused  me  to  seek  help  from  a  professional  person, 
such  as  a  psychiatrist,  doctor,  clergyman,  psychologist, 
or  social  worker. 

2.  The.  color  of  the  ball  in  the  window  is  blue, 

The  emotional  problem  trial  was  different  from  the  others  in  that  it  used 
three  colors  of  balls:  red,  white,  and  blue.  After  shaking  the  box  In  the 
usual  fashion  and  causing  a  ball  to  roll  into  the  window,  respondents  with  a 
red  ball  in  tho  window  answered  the  emotional  problem  statement.  Those  with 
either  a  white  or  blue  ball  in  the  window  answered  the  other  statement. 
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Estimates  of  the  proportion  of  women  with  an  emotional  problem  showed 
that  22.8  per  cent  had  experienced  such  a  problem  which  required  professional 
attention.  The  estimates  were  higher  for  nonwhites  (30. 6%)  then  for  whites 
(19. OX),  he  with  abortion  and  oral  contraceptives ,  the  emotional  problem 
estimates  were  computed  under  the  assumption  that  the  value  of  the  unrelated 
question  for  the  group  was  known  la  advance.  This  value  for  the  emotional 
problem  aspect  of  the  study  was  defined  as  the  ratio  of  the  number  of  women 
drawing  a  blue  ball  to  the  number  drawing  a  non-red  ball.  The  exact  value  of 
the  numerator  and  denominator  were,  of  course,  unknown.  The  expected  value  of 
this  proportion  is  the  ratio  of  blue  balls  to  non-red  balls  in  the  randomizing 
device,  and  this  was  known.  It  wm  considered  to  be  a  reasonable  estimate  of 
the  proportion  answering  "Yes"  to  the  statement,  "The  color  of  the  ball  in  tha 
window  is  blue"  and  was  used  in  calculations  of  emotional  problem  estimates. 

The  North  Carolina  Abortion  Study  also  collected  quantitative  data  on 
abortion  and  income  using  the  randomized  response  procedure.  The  statements 
on  abortion  were  as  follows: 

1,  How  many  abortions  have  you  had  during  your  lifetime? 

2.  If  a  woman  has  to  work  full-time  to  make  a  living,  how 
many  children  do  you  think  she  should  have? 

■i 

The  principal  difference  between  thlB  application  of  the  technique  and 
the  situations  previously  described  is  that  the  answer  was  expected  to  be  a 
small  nunber  rather  than  a  "Yes"  or  "No"  response.  The  estimation  procedure 
was  entirely  different,  of  course.  The  Income  statements  were  completely 
analogous  to  those  for  number  of  abortion;  in  both  application  and  analysis. 
They  wore: 


1.  About  how  much  money  In  dollars  did  the  head  of  this 
household  earn  last  year? 

2.  About  how  much  money  In  dollars  do  you  think  the  Kverfcfee 
head  of  a  household  of  your  else  earns  In  a  year? 

Estimates  and  variances  of  tho  mean  number  of  abortions  obtained  over  a 
lifetime  in  an  urban  population  of  women ,  and  of  mean  income  of  heeds  of 
households,  were  reported  In  [8].  A  more  detailed  treatment  of  data  collected 
In  the  North  Carolina  Abortion  Study  and  the  analysis,  both  qualitative  and 
quantitative,  may  bs  found  iri  [9,10], 

Other  applications.  Proctor  and  Gamble,  Ltd.,  conducted  a  survoy  In 
England  in  1968  using  the  randomized  response  technique  to  gain  informa¬ 
tion  on  personal  hygiene  habits  of  housewives.  Two  sets  of  questions  were 
used  with  the  plastic  box  technique  previously  described: 

Box  1 

1.  Have  you  cleaned  your  teeth  today? 

2;  Were  you  born  in  one  of  the  following  months: 

Januury,  February,  March,  April,  May,  June,  or  July? 

Box  2 

1.  Have  you  had  a  bath  in  the  last  two  days? 

2,  Were  y  >u  bom  on  one  of  the  following  days  of  the  month: 

1st,  2nd,  3rd,  4th,  5th,  6th,  7th,  8th,. 9th,  10th,  11th, 

12  th? 

For  purposes  of  comparison,  each  of  the  two  "sensitive"  questions  above 
was  also  ssked  through  the  direct  question  approach  of  sn  analogous  group  of 
women.  The  two  sets  of  estimates  of  the  proportion  of  housewives  cleaning 
their  teeth  agreed  within  limits  of  sampling  error.  The  same  was  true  of  the 
estimates  pertaining  to  having  a  bath  in  the  past  two  days.  The  authors  con- 


eluded  that  the  housewives  understood  what  was  required  of  them  in  the  randomised 
response  application,  but  that  neither  "sensitive"  question  proved  to  be 
embarrassing  to  the  housewives.  Hence,  they  felt  the  randomised  response 
was  unnecessary  as  long  *• direct  questioning  is  not  embarrassing  to  the 
respondent. 

Barksdale  reported  using  the  randomised  response  procedure  among  selected 
students  at  the  University  of  North  Carolina^  to  justify  the  feasibility  of 
some  of  the  extensions  to  the  method  which  he  recommended  aa  a  result  of  hie 
research.  He  was  concerned  with  determining  the  proportion  of  students  who 
had  smoked  marijuana  at  some  time  in  their  lives ,  and  the  proportion  who  had 
never  cheated  on  an  examination.  This  survey  wss  beset  with  many  unforeseen 
problems  leading  the  author  to  conclude  that  "due  to  the  lack  of  an  adequate 
aample  size  and  experimental  procedure,  little  credibility  can  be  aaaodatod 
with  the  observations". 

The  Research  Triangle  Institute  used  the  technique  in  1970  in  a  drinking- 
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driving  attitude  survey  in  Mecklenburg  County,  North  Carolina1  .  Comparisons 
wsra  made  between  randomized  response  and  direct  question  responses. 

LESSONS  LEARNED  FROM  FIELD  EXPERIENCES 

The  randomized  response  technique  has  now  been  widely  tested  In  the  field 
and  much  has  been  learned  as  a  result  of  these  valuable  experiences.  Some  of 
the  additional  knowledge  gleaned  from  these  surveys  is  of  a  positive  nature 
which  might  be  implemented  immediately  to  improve  estimates  of  the  sensitive 
characteristic.  The  field  experiences  also  identified  problems  in  the 
procedure  which  are  not  readily  amenable  to  solution,  and  which  require  further 
research.  Some  of  the  more  Important  lessons  gained  from  field  experiences  arc 
enumerated  below. 
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1.  The  respondent  must  be  convinced  that  the  survey  is  for  legitimate 
research  purposes,  and  that  his  anonymity  to  the  aensitlve  question 
cannot  and  will  not  be  violated.  The  institution  with  which  the 
Interviewer  identifies  himself  is  of  particular  importance  in  that 
the  respondent  must  not  get  the  impression  that  some  agency  is  prying 
into  his  personal  life  for  some  ulterior  motive,. 

2.  Estimates  of  the  parameter,  jr^,  and  Its  variance  are  extremely  sensitive 
to  the  P^  (und  P^)  values  (probabilities  of  selection  of  the  sensitive 
question  associated  with  the  randomization  device).  It  is,  therefore, 
Imperative  that  the  device  in  practice  produce  these  values  within  reasonable 
sampling  error  and  untainted  by  non-sampling  error  and  bias. 

3.  Further  research  should  be  directed  toward  determining  optimum  P^ 

(end  Pg)  values  from  the  point  of  view  of  acceptability  to  the  respondent. 

4.  Intimates  of  the  parameter,  ir^,  and  its  variance  are  sensitive  to  tha 
choice  of  the  alternate  (or  non-aensitlve)  question.  The  latter  should 
be  chosen  with  known  and  relatively  low  prevalence  in  the  population 

being  surveyed.  It  should  not  be  foreign  to  the  group,  and  must  be  culturally 
acceptable.  It  is  best  to  choose  in  the  anticipated  region  of  ir^.  If  these 
conditions  cannot  be  met,  consideration  should  be  given  to  formulating 
the  alternate  question  as  a  function  of  the  randomization  device  (aa 
was  done  in  the  emotional  problem  trial). 

5.  The  interviewer  should  thoroughly  understand  the  technique  and  be  able 
clearly  and  lucidly  Vo  impart  this  knowledge  to  a  respondent  whose 
education  is  minimal.  The  interviewer  must  evoke  a  feeling  such  that 

tha  respondent  has  confidence  in  her  integrity  and  sincerity.  Furthermore, 
the  interviewer  should  have  faith  in  the  procedure  and  not  be  skeptical 
lest  this  attitude  be  unconsciously  transmitted  to  the  respondent. 


1.  The  respondent  must  be  convinced  that  the  survey  is  for  legitimate 
research  purposes,  and  that  his  anonymity  to  the  sensitive  question 
cannot  and  will  not  be  violated.  The  Institution  with  which  the 

interviewer  identifies  himself  is  of  particular  importance  in  that 

...  ... 

the  tes^gfent  roust  not  gat  the  impression  that  some  agency  is  prying 
into  his  personal  life  for  some  ulterior  motive,. 

2.  Estimates  of  the  parameter,  n^,  and  its  variance  ara  extremely  sensitive 
to  the  (und  P^)  values  (probabilities  of  selection  of  the  eanaitlva 
question  associated  with  the  randomization  device).  It  is,  therefore, 
Imperative  that  the  device  in  praoti?a  produce  theae  values  within  reasonable 
sampling  error  and  untainted  by  non-sampling  error  and  bias. 

3.  Further  research  should  be  directed  toward  determining  optimum 

(and  Pg)  values  from  the  point  of  view  of  acceptability  to  the  respondent. 

4.  Estimates  of  the  parameter,  ir^,  and  its  variance  are  sensitive  to  the 
choice  of  the  alternate  (or  non-seneltivo)  question.  The  letter  should 
be  chosen  with  known  and  relatively  low  prevalence  in  the  population 

being  surveyed.  It  should  not  be  foreign  tu  the  group,  and  must  be  culturally 
acceptable.  It  is  boot  to  chooso  ir^  in  the  anticipated  region  of  rr^.  If  these 
conditions  cannot  be  met,  consideration  should  be  given  to  formulating 
the  alternate  question  as  s  function  of  the  randomization  device  (as 
waa  done  in  the  emotional  problem  trial). 

5.  The  interviewer  should  thoroughly  understand  the  technique  and  be  able 
dearly  and  lucidly  to  Impart  this  knowledge  to  a  respondent  whose 
education  is  minimal.  The  interviewer  must  evoke  a  feeling  Buch  that 

the  respondent  has  confidence  in  h6r  integrity  end  sincerity.  Furthermore, 
the  interviewer  should  have  faith  in  the  procedure  and  not  be  skeptical 
lest  this  attitude  be  unconsciously  transmitted  to  the  respondent. 
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6.  Tha  ptoblan  of  administering  tha  tachnlque  to  lllltarataa  oust  bo 
resolved. 

7.  Tha  technique  should  not  be  used  In  situations  where  the  direct  question 
can  be  used. 

8.  Research  toward  a  more  efficient  randomisation  device  ahould  continue. 
Making  P  equivalent  to  y  hae  intuitive  appeal  and  would  tend  to  lessen 
suspicion  on  the  part  of  respondents  that  the  procedure  was  rigged  in 
some  way.  Tills  value  cannot  be  used  in  the  original  Warner  modal  as 
indicated  in  Equation  (1) $  it  can  be  used  in  the  unrelated  question 
model  (Equation  2)  but  with  a  lose  in  efficiency  when  compared  with 
higher  values  of  P.  This  loss  might  possibly  be  overcome  by  increased 
cooperation  on  the  part  of  respondents  when,  for  example,  e  simple  tosa 
of  a  coin  is  used  In  lieu  of  the  box-and-balls  device  currently  used. 

POSSIBLE  APPLICATIONS  IN  MILITARY  RESEARCH 

There  are  several  sensitive  areas  involving  the  military  in  which  informa¬ 
tion  may  be  vital  but  unavailable  because,  for  obvious  reasons,  the  direct 
question  approach  would  not  yield  truthful  responses. 

One  such  matter  involves  the  usa  of  drugs  by  members  of  the  armed  forces. 
The  randomized  response  method  could  be  utilized  to  determine  the  proportion  of 
returning  personnel  or  veterans,  for  example,  who  experimented  with  heroin, 
or  any  of  the  other  drugs  available  during  their  service  abroad.  Such  e 
study  would  permit  estimates  of  drug  use  by  members  of  the  military  classified 
by  rank,  branch  of  service,  location  of  station,  age,  length  of  service,  race, 
and  a  multitude  of  other  characteristics. 

Homosexual  practices  among  the  military  is  another  field  in  which  reliable 
information  may  be  practically  non-existent.  It  is  generally  conceded  that 
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knowledge  of  the  extent  of  a  problem  is  a  prerequisite  to  launching  a  program 
to  prevent  or  reduce  the  problem.  This  would  be  particularly  true  for  homosexual 
or  other  deviant  sexual  practices  that  may  exist  in  the  armed  forces.  Detailed 
classification  by  branch,  rank,  and  other  strata  would  also  be  available. 

A  study  of  attitudes  and  opinions  of  members  of  the  armed  forces  toward 
officers,  military  service  In  general,  discipline,  and  related  matters  is 
another  fertile  field  for  exploration  with  the  randomized  response  technique. 

Alcoholism  among  the  military  is  yet  another  .field  in  which  the  lack  of 
valid  data  on  prevalence  could  be  rectified  through  the  use  of  randomized 
response.  Obviously,  there  are  many  other  areas  of  potential  use  in  the 
military  which  could  be  enumerated  but  those  mentioned  suffice  to  indicate 
the  potential  of  the  method. 

SUMMARY 

A  method  of  reducing  or  eliminating  bias  introduced  when  respondents  in 
surveys  deliberately  give  false  Information,  or 'refuse  to  answer  questions 
of  a  sensitive  nature,  is  described.  The  method  is  known  as  randomized 
response. 

The  growth  and  development  of  the  method  since  ItB  origin  in  1965  to 
the  present  time  is  documented.  Special  emphasis  1b  placed  on  lessons  that 
have  been  learned  through  several  field  experiences  involving  application  of 
the  technique.  Some  further  research  on  the  methodology  ia  indicated. 

Several  potential  applications  of  randomized  response  among  the  armed 
forces  are  discussed.  These  Include  the  use  of  drugs,  homosexual  practices, 
attitudes  or  opinions  toward  military  matters,  and  alcoholism. 
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THE  CHARACTERIZATION  AND  ANALYSIS  OF 
COMPLEX  BIOLOGICAL  RESPONSES 

Captain  J.  Richard  Jennings 
Dept.  of  Experimental  Psychophysiology 
Walter  Reed  Army  Institute  of  Research 
Washington,  D.  C, 


ABSTRACT.  Complex,  time-varying  biological  responses  to  discrete  psychologies 
events  pose  problems  of  numerical  characterisation  add  atatiatical  analysis. 
Biological  responses,  such  as  brief  heart  rate  changes  and  cortical  evoked 
potentials,  are  multiphasic  response  with  inherent  dependency  within  the 
response.  T|yi.»  dependency  should  be  identified  in  order  to  separate 
biological  control  influences  from  the  influence  of  the  psychological 
event.  A  prototype  exporiment  is  developed  based  on  actual  data  illustrating . 
these  problems.  Alternate  analyses  are  suggested  and  advice  concerning  the 
application  of  multivariate  techniques  is  sought. 

INTRODUCTION .  The  accurate  and  complete  numerical  characterisation  of 
a  biological  system's  pesponse  to  stimulation  is  a  vexing  problem  for  many 
investigators.  This  is  particularly  true  of  the  investigator  seeking  relations 
between  psychological  and  physiological  functioning  within  a  normal  human 
being.  In  such  investigations  a  clearly  specified  psychological  event  is 
presented  and  the  physiological  reaction  is  observed.  The  physiological 
reaction  is  not  simple,  however.  Reactions  are  generally  extended  in  time  and 
vary  during  this  time.  In  addition,  such  physiological  reactions  are  multi- 
determined  —  biological  controls  on  the  response  necessarily  interact  with 
the  reaction  to  the  psychological  event.  Current  knowledge  of  these  response 
determining  events  and  their  interactions  doesn't  seem  to  allow  any  straight¬ 
forward  deterministic  modeling  of  the  biological  response  process.  The 
first  approximation  to  understanding  .these  response  processes  must  be,  It 
seems,  through  an  empirical  and  statistical  approach. 

The  biological  responses  of  particular  interest  are  those  which  occur  as 
time-varying  responses  to  relatively  specific  stimulation.  An  exampls  of 
such  a  response  would  be  a  second  to  second  change  in  heart  rate  induced 
by  a  burst  of  white  noise  or  an  electrical  potential  change  in  brain  waves 
elicited  by  a  flash  of  colored  light.  To  elaborate  the  heart  rate  example, 
at  the  onset  of  a  burst  of  white  noise,  a  person's  heart  rate,  which 
even  without  stimulation  shows  momentary  variations  in  rate,  may  momentarlxy 
slow  down  and  then  speed  up  prior  to  returning  to  a  pre-stimulation  rate. 

With  this  type  of  response  in  mind,  the  basic  questions  of  this  paper  can 
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be  poiied t  "How  can  this  biological  response  be  characterised  in  a  simple 
yet  complete  manner?  How  can  the  characterised  response  be  compared  to 
both  pre-ctimulus  variability  and  responses  to  other  psychological  stimuli?" 

A  PROTOTYPE  EXPERIMENT.  A  somewhat  simplified  prototype  experiment  has  been 
designed  that  illustrates  a  general  approach  to  experimentation  wlttv 
psychophysiology.  After  the  description  of  this  prototype  experiment,  the 
problem  will  be  reduced  to  the  comparison  cf  two  heart,  rate  response  curve e 
based  on  reel  data  collected  under  circumstances  analogous  to  those  of  the 
prototype  experiment..  -  •  . 

.  The  prototype  study  concerns  the  information  processing  of  numbers 
within  s  map  raading  problem.  Subjects  highly  familiar  with  a  certain  map 
wars  given  two  tusks.  Tor  both  tasks  s  ait  of  sight  numbers  raprassnting 
map  grid  coordinates  was  presented  visually  for  five  asconda  using  a  slida 
projector.  In  one  task  the  subject  was  required  to  remember  the  exact 
coordinates.  In  the  other  task  the  subject  was  asked  only  to  decide  which 
quadrant  of  the  map  contained  the  coordinates  displayed.  The  physiological 
concomitants  of  performance  in  these  two  teaks  were  of  primary  interest . 

More  specifically,  a  pronounced  physiological  reaction  waa  expected  to 
accompany  the  task  demand  for  exact  grid  specification,  but  not  to 
accompcny  the  leas  demanding  quadrant  placement  task. 

The  experimental  design  specified  the  uee  of  ten  subjects.  In  order  to 
use  each  subject  as  hie  own  control,  each  subject  performed  in  both  taake. 
Five  subjects  were  randomly  grouped  to  receive  the  coordinate  memory  task 
first  and  the  remaining  five  received  the  quadrant  placement  teak  first. 

Three  reaponaa  measures  were  collected i  heart  rate,  skip  conductance  (GSR's) 
and  accuracy  of  performance  on  the  information  processing  teaks.  Each 
task  waa  composed  of  ten  trials  using  different  map  coordinates.  For  each 
task  the  reaponaa  measures  were  averaged  over  the  ten  trlale .  The  design  can 
be  diagrammed  as  follows: 
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values  over  ten  trials.  Subscripting  is  represented  by  Xinjp,  where  i 
represents  the  2  task  orders,  n  represents  the  10  subjects,  j  represents  the 
2  tasks,  and  p  represents  the  3  response  variables. 


Assuming  for  a  moment  that  the  three  response  measures  ware  simply  single 
numbers  reflecting  the  heart  rata,  akin  conductance,  and  performance  responses, 
a  rachar  straight-forward  analysis  of  the  results  might  ba  possible.  A  tech¬ 
nique  comparable  to  the  profile  multivariate  analysis  of  varieties  suggested 
by  Morrison  (1967)  might  provide  an  overall  estimate  of  the  preaence  of  any 
significant  effect  due  to  teak  and  order  treatments  within  the  three  response 
mesauras.  Subsequent  comparisons  following  a  multivariate 
determine  the  locus  of  any  significant  affects.  The  applicatl68ruC  fni 
technique, to  the  current  data  la  not  straight-forward.  However,  'papers  such 
as  Pottholf  &  Roy  (1964)  suggast  that  ganerallaatlon  to  cases  such  as  the 
current  one  are  possible.  In  short,  if  the  data  coitld  truly  be  fit  into  the 
layout  of  fable  1,  a  comprehensive  statistical  analysla  would  seem  possible. 

The  problem  arises  that,  in  fact,  at  least  two  of  the  response  measures 
(heart  rate  and  GSR)  era  not  single  numbers  but  a  vector  of  dependent  values 
reflecting  both  pre-  and  poat-stimulus  physiological  activity.  The  decision 
of  how  to  analyse  this  vector  of  values  must  ba  baaed  on  some  knowledge  of 
the  nature  of  the  physiological  response  and  also  upon  the  validity  and 
availability  of  atatiatlcal  methods  of  rapraaanting  this  data.  A  detailed 
knowledge  of  tha  heart  rata  rasponaa  measure  may  aid  in  decision, 

THE  HEART  RATE  RESPONSE.  Psychophyaiologista  find  it  meaningful  to  measure 
heart  rate  during  psychological  performance  because  of  a  belief  that  the 
autonomic  nervous  system  is  continuously  adjusting  the  Internal  mlleau  of 
the  organism  on  the  basis  of  events  in  the  external  environment.  These 
adjustments  are  generally  not  massive  enough  to  .enter  our  awaranesa,  although 
in  states  such  as  rage  we  are  aware  of  physiological  changes.  Minor 
adjustments  occur,  howsver,  and  can  consistently  be  related  to  psychological 
events.  Thasa  adjustments  appear  as  momentary  changes  that  are  not 
obasrvad  if  pulse  rate  is  measured  over  a  minute  or  so.  In  order  to 
observe  such  changes,  the  time  between  individual  beatB  of  the  heart  must  be 
measured.  This  time  can  be  converted  to  the  familiar  beats  per  minute  (bpm). 


Heart  Rate  (bpm)  ■  60  seconds 

Inter-beat  interval  (seconds) 

For  example  any  interbeat  interval  of  one  second  yields  a  heart  rate  of  60 
beats  per  minute.  In  such  a  manner  instantaneous  heart  rates  can  be  expressed 
for  individual  beats  before  and  after  stimulation.  In  Figure  1  the  rate  of 
the  heart  Is  plotted  for  six  beats  prior  to  and  six  beats  following  stimulus 
presentation.  Focusing  on  the  curve  with  the  solid  line,  the  first  three 
beats  plotted  show  a  stable  heart  rate  of  82  beats  per  minute.  This  is 
followed  by  an  increase  to  83  bpm  for  the  next  three  beats  and  then  the 
reaction  to  tha  stimulus  follows,  a  decrease  in  rate  followed  by  s  substan¬ 
tial  increase,  These  are  the  sorts  of  brief  changes  in  heart  rate  under 
study.  Note  that  Figure  1  represents  a  total  time  of  about  ten  seconds. 

Returning  to  the  prototype  experiment.  Figure  1  represents  the  cardiac 
responses  to  the  two  experimental  tasks.  The  heart  best  response  curves  are 
compared  for  the  coordinate  memory  task  and  the  quadrant  placement  task  (the 
order  of  task  presentation  Is  ignored  for  sake  of  simplicity).  The  data 
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Heart  rate  response  data  for  the  two  information  processing  tasks* 
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for  the  Individual  aubjecta  are  shown  in  Table  2.  Figure  1  indicate*  that 
during  the  coordinate  memory  task  a  relatively  alow  heart  rate  prior  to 
atimulatlon  waa  followed  by  a  brief  alowing  and  than  a  substantial  speeding 
after  atimulatlon.  For  the  quadrant  placement  teak  a  relatively  fast  heart 
rate  waa  replaced  by  a  progressively  alowing  heart  rate  after  atimulatlon. 

The  basic  quaations  of  response  characterisation  and  analysis  can  be 
posed  based  upon  the  response  curves  shown  in  Figure  It 

1.  Does  the  post  stimulus  reaction  in  either  teak  constitute  a  reel 
response  to  atimulatlon  as  opposed  to  a  chance  occurrence? 

2.  Given  a  true  change  in  heart  rate,  do  the  complex,  time  varying 
responses  in  Figure  1  represent  a  unitary  reaction  of  the  cardiac  control 
system?  Alternately,  can  this  reaction  be  meaningfully  separated  into  a 
number  of  Independent  or  aemi -dependent  components  representing,  perhaps, 
the  influence  of  different  cardiac  control  mechanisms? 

3.  Given  a  set  of  meaningful  components,  can  a  comparison  serosa  taska 
be  made  of  these  components? 

‘i 

None  of  these  questions  are  novel.  Various  approaches  to  their  solution 
have  been  offered,  but  none  have  been  generally  accepted.  Furthermore 
current  interest  in  multivariate  techniques  leads  to, /a  reopening  of  such 
questions  in  the  lijht  of  the  availability  of  these  methods.  The  eventual 
goal  being  a  simultaneous  analysis  of  a  whole  set  of  biological  and  psycho¬ 
logical  dependent  variables. 

SOME  APPROACHES  TO  SOLUTIONS.  Question  It  Does  the  post  stimulus  reaction  in 
either  task  constitute  a  real  response  to  stimulation  as  opposed  to  a  chance 
occurrence? 

A  relatively  straightforward  solution  to  this  problem  may  exist.  The 
problem  is  essentially  to  decide  whether  two  vectors  both  of  size  n  (the 
baseline  beats  and  the  response  beats)  are  significantly  different.  Given 
adequate  experimental  planning,  an  equal  number  of  heart  beat?  should  be 
available  before  and  after  stimulation.  The  pre-stimulation  values  should 
be  collected  during  a  time  period  adjacent  to  the  response  when  no  psycho¬ 
logical  events  are  Influencing  the  heart  rate  consistently.  This  la  achieved 
by  Initiating  stimulus  events,  e,g.  map  coordinates,  at  quasi-random  times 
after  the  reaponso  to  the  previoue  event.  Thus  the  stimulus  induced  pertur¬ 
bation  in  cardiac  rate  may  be  compared  to  a  sample  of  normally  varying 
cardiac  rate. 

Two  requirements  muet.be  placed  on  the  analysis.  First,  it  must  take 
Into  account  the  marked  dependency  between  beats  end  between  the  two  periods. 
In  addition  the  integrity  of  the  beats  must  be  maintained  —  averaging  across 
beats  would  eliminate  the  transient  response  that  is  under  study. 
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Two  statistical  approaches  have  baan  used  both  of  which  at as  inelegant 
if  not  unaatiafactory.  One  approach  (a.g,  Coqusry  6  Lacey ,  1966)  haa  been 
to  do  a  aeries  of  univariate  comparisons  (e.g,  Wilcoxon  paired  comparison  or 
t-taat  for  dependent  x'a)  between  analogous  beats  (pre-stimulus  beet  1  against 
post-stimulus  beat  1,  etc.).  This  approach  yields  a  set  of  points  in  the 
response  period  that  are  supposedly  significantly  different  from  analogous 
points  in  the  pre-period.  This  approach  would  seam  to  capitalise  on  the 
beat-to-beat  dependency  within  the  data  as  well  as  on  random  variations  in 
the  numerous  errors  terms  used  in  the  t-teata. 

A  second  approach  (e.g,  Wilson ,  1967  aa  derived  from  Greenhouse  & 

Geisser,  1959)  uses  an  analysis  of  variance  design  which  treats  beata  as  a 
classification  factor.  In  the  case  of  the  current  data,  a  two-way  analysis  of 
variance,  might  be  set  up  with  pre-  vs,  post-stimulation  as  a  factor  (two 
levels)  and  beats  as  factor  (6  levels).  A  slgnif leant  main  affect  of  beets 
is  Interpreted  as  showing  that  consistent  differences  between  beata  occur. 
Central  interest  is,  however,  in  the  beats  by  pre-  va.  post-interaction. 

This  interaction  is  Interpreted  as  showing  that  the  beat-by-beat  response 
curves  are  different  between  the  pre-  and  post-stimulation  period.  Despite 
adjustments  in  degrees  of  freedom  to  "account  for"  dependency,  this  sort 
of  analysis  seems  questionable  if  only  because  of  the  peculiar  use  of  beats 
as  a  classification  factor.  Another  problem  with  this  analysis  may  be  the 
varying  degrees  of  dependency  often  found  in  physiological  data. 

The  direction  of  solution  for  this  problem  seems  at  present  to  be  a 
multivariate  analysis  comparing  the  two  vectors  simultaneously.  The 
problem  of  comparing  dependent  vectors  within  the  same  sampling  unit  remains 
somewhat  confusing,  however.  The  intuitive  appeal  of  the  multivariate 
approach  Is  the  consideration  of  heart  beats  as  a  multiple  response  rather 
.  than  a  classification  factor.  The  question  remsins  whether  the  underlying 
model  truly  represents  ths  data  or  questions  better  than  the  univariate 
approach.  Aa  noted  previously,  papers  such  as  that  by  Fotthoff  and  Roy 
(1964)  suggest  aome  optimism  on  this  account. 

Question  2s  Given  a  true  change  in  heart  rate,  do  the  complex,  time 
varying  responses  in  Figure  1  represent  a  unitary  reaction  of  the  cardiac 
control  system?  Alternately,  can  this  reaction  be  meaningfully  separated 
into  a  number  of  independent  or  aeal-depandent  components  representing,  perhaps 
the  influence  of  different  cardiac  control  mechanisms? 

If  the  cardiac  response  la  indeed  different  from  baseline  activity,  the 
nature  of  this  response  becomes  the  next  question.  Has  the  coordinate  memory 
task  produced  a  truly  independent  accaleratory  change  in  heart  rate  or  is  the 
speeding  seen  in  the  latter  part  of  the  response  period  due  to  biological 
regulatory  adjustment  from  the  immediately  prior  slowing  —  or  even  due  to 
the  relatively  low  pre-stimulation  heart  rate?  In  short  what  is  the  depen¬ 
dency  structure  within  the  total  HR  sample  including  baseline  and  response. 


An  axampla  of  tha  typo  of  biological  control  system  may  be, helpful  In 
explaining  tha  Mod  to  .JsaW'kaTy  of  changes  in  rsaponss  that  srs  physiologi¬ 
cally  induced  rathar  chan  caused  by  stimulation.  This  example  will  also 
demonstrate  the  need  to  look  within  tha  HR  response  for  dependency.  A  basic 
and  well  known  control  mechanism  present  in  biology  is  the  homeostatic  loop. 

In  such  cases  differences  from  a  preset  internal  state"  of  the  system  are 
corrected  by  a  negative  feedback  loop  based  on  the  output  of  the  system.  In 
the  cardiac  system,  for  example,  barorecaptors  in  major  arteries  sense  pressure 
and  rate  changes.  Significant  changes  in  rets  or  pressure  cause  the  baro- 
raceptor  to  send  via  the  nervous  system  inhibitory  signals  which  result  in  a 
corrective  return  of  rate  end  pressure  to  normal  levels.  Thus,  an  efficient 
control  loop  prevents  any  major  changes  in  cardiac  pataaetars. 

Within  such  a  system,  any  change  in  heart  rata  would  be  expected  to  be 
followed  by  a  feedback-induced  changa  in  tha  opposite  direction.  To  tha 
paychologist  thle  feedback  induced  changa  is  of  little  interest,  but  rathar 
may  mask  to  some  degree  further  cardiac  changs  induced  by  the  experimental  task. 
Thus  s  brief  slowing  ofH&art  rata  (as  in  the  coordinate  memory  raeponea 
curve)  may  elicit  a  homeostatic  acceleration.  This  acceleration  may  either 
■ask  further  deceleration  of  heart  rate  or  combine  with  a  psychologically- 
induced  acceleration  thereby  confounding  this  raectlon.  Thus,  if  one  wishes/ 
to  demonstrate  an  accelatory  reaction  to  a  task,  the  independence  of  the 
acceleratory  reaction  vls-a-vla  prior  changes  must  be  demonstrated.  Prior 
changes  must  include  both  pre-stimulation  oscillation  as  well  as  preceding 
changes  within  the  cardiac  response. 

Psychologists  have  bean  concerned  about  biological  control  influences  or 
reactions  for  at  least  20  years.  Early  papers  in  the  field  formulated  con¬ 
cepts  such  as  the  "Lew  of  Initial  Values"  stating  that  the  preresponsa  level 
of  tha  variable  influenced  the  rasponas  level.  In  general  the  notion  was  that 
the  lower  the  initial  level  the  greater  the  probability  of  a  large  response. 

In  practice,  moot  have  followed  the  genarsl  outline  of  the  suggestion  of 
Lacey  (1956)  and  empirlcaWy  found  tha  correlation  between  pre-  end 
post-stimulation  levels  "end  coverisd  out  this  Influence  upon  the  response. 

Let  us  look  at  the  application  of  such  notions  to  our  HR  case. 

The  problem  of  the  nature  of  the  beat-by-baat  data  rises  at  once,  Tha 
data  ia  collected  in  order  to  look  at  momentary  raactiona  not  at  changaa 
from  a  base  laval  to  a  raaponae  level.  Through  averaging  across  beats  the 
HR  data  may  be  forced  into  thla  model,  but  thara  Is  no  rstlonale  for  doing 
this.  The  underlying  rstlonale  for  correcting  for  dependency  is  to  subtract 
out  or  at  least  be  aware  of  biological  control  influsness  upon  tha  response 
unrelated  to  tha  experimental  manipulations  under  study.  The  response 
asaumedly  due  to  tha  experimental  manipulation  is  a  moment!  ry  flunctuation 
and  thua  measures  of  this  fluctuation  not  averages  should  be  corrected.  In 
like  manner  it  seems  more  reasonable  to  covary  with  the  pre-stimulus  degree 
of  fluctuation  than  to  covary  with  tha  pre-stimulus  average. 

Tha  influence  upon  one  another  of  fluctuations  within  the  response  itself 
also  ahobld  be  investigated.  Each  teak  is  expected,  however,  to  produce  s 
distinctive  response  curve.  For  example,  the  response  to  the  coordinate 


memory  task  includes  an  early  deceleration  while  quadrant  placement  task 
does  not  show  deceleration  until  lata  In  the  response  period.  The  maximum 
heart  rata  in  the  coordinate  memory  teak  response  should  be  corrected  for 
the  influence  of  prior  deceleration.  It.  is  not  appropriate,  however,  to 
perform  the  seas  correction  on  tha  quadrant  placement  data  or  fhdaad  on  tha 
combined  quadrant  placement  and  coordinate  memory  .data.  In  contract  the 
correction  for  pre-stimulation  variability  should  be  applied  across  all  eaake, 
Pre-atlmulua  variability  la  assumed  f:>  ',«  inherent  variability  and  every 
pre-stimulus  period  should  be  a  sutpl*  of  this  variability.  Thus  In  order 
to  correct  for  the  influence  of  the  Inherent  variability,  the  best  estimate 
would  be  the  mean  of  all  the  pre-stimulus  periods, 

"'Che  upshot  of  such  thoughts  aaems  to  bs  an  extensive  correlational 
analysis  of  tha  complete  HR  (baseline  and  response)  curve.  Two  analyses  are 
suggested!  1)  determine  tha  correlation  between  pre-  and  pcst-atlmulus 
fluctuation  and  then  t*»  xemovs  the  Influence  of  the  pre-stimulus  fluctuation; 
2)  determine  the  etew Sgfere  of  tha  pose-stimulation  rwaponsa  separately  for 
each  experimental  treatment.  The  first  enalysas  would  involve  correlations 
across  subjects  and  across  all  experimental  treatments  while  the  second  could 
be  dona  within  and/or  acrose  subjects  within  an  experimental  treatment  group. 

Buth  of  the  suggested  analyses  .require  that  a  score  or  eat  of  score* 
representing  HR  change  be  derived  from  the  data.  These  scores  might  be 
date  points  themselvss  or  a  derlvsd  statistic.  Initially  on*  might  taka 
points  of  intuitlva  intarest  such  as  maxima  and  minima  and  treat  these  aa 
scores  representing  change.  Two  advantages  of  such  scores  can  bs  cited. 

Piret,  they  estimate  tha  fluctuation  in  tha  date  without  losing  the  identity 
of  the  maximum  and  minimum  as  would  be  don*  with  a  variance  estimate. 

Second,  they  express  the  variability  independently  of  the  time  dimension. 

This  second  advantage  ie  important  within  the  pre-period  because  this  period 
is  usually  initiated  randomly  with  respect  to  time.  Thus  the  pre-period  will 
ahow  random  segments  of  any  regular  periodicity.  With  respect  to  the 
poat-atlmulatlon  response,  the  temporal  information  becomes  important  in 
defining  the  reeponae,  and  thua  a imp la  maxima  and  minima  are  not  as  advanta¬ 
geous.  In  cases  where  the  poet-stimulation  response  is  not  precisely 
time-locked,  however,  the  use  of  maxltnA  and  minima  may  be  preferable  to  the 
method  to  be  developed  below.  . 

The  correlation  of  maximum  and  minimum  acoree  from  the  pre  and  reeponae 
periods  could  serve  to  estimate  the  degree  to  which  the  mXxima  end  minima  of 
the  response  reflect  baseline  fluctuations.  This  asssssment  may  bs  made 
aeperately  for  the  basal  maxims  and  ths  basal  minima's  effset  on  rasponae 
maxima  and  minima.  Ragrassion  aquations  basad  on  ons  or  mors  of  ths  four 
corralations  computed  could  ba  used  to  "correct"  the  response  scores  by  the 
baseline  scores.  This  proesdurs  appears  to  be  a  workable  solution  to  the 
"initial  value"  problem  albeit  a  cumbersome  solution.  Obj actions  to  this 
solution  ars  the  Intuitive  approach  of  looking  at  peeks  and  valleys  and  the 
probable  unreliability  of  single  points  plucked  from  either  the  basal  or 
response  curves. 
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An  alternative  to  the  intuitive  points  of  interest  would  be  an  attempt 
to  incorporate  ell  the  heart  beets  Into  the  analysis.  This  would  in  general 
mean  doing  analyses  which  preserve  the  time  dimension,  A  simple  correlation 
between  analogous  points  in  pre-  and  post-stimulation  response  might  be 
considered*  The  random  timing  nature  of  the  pre-stimulation  period  discussed 
above  argues  against  this  approach  for  the  pre-  vs,  post-reapoise  comparison, 
8osm  form  of  time  independent  data  representation,  such  as  developed  above, 
would  seam  to  be  the  preferred  approach  to  the  pre-poet  comparison.  An 
alternative  night  be  to  eliminate  the  specific  pre-period  lh  favor  of  e 
eubitentiatl  baseline  period  of  ten  minutes  or  so.  Auto-correlational  or 
time  aerlaa  techniques  could  ba  applied  to  the  baseline  data  to  derive  the 
componenta  of  the  raating  variability.  The  reaponae  curves  could  then  be 
compared  to  the  baaeline  spectra.  Thia  comparison  would  probably  not  be 
statistical,  however. 

The  dependency  structure  withl.i  the  cardiac  roaponsa  to  a  specific  teak 
(the  second  analysla)  may  be  untangled  with  e  time-based  correlational  analysis. 
The  fruitful  application  of  this  analysis  requires  that  tha  cardiac  response 
to  the  teak  be  time-locked  in  a  similar  manner  across  all  aubjacts.  If  thia 
is  the  case,  a  simple  correlation  matrix  consisting  of  each  heart  beet's 
correlation  with  the  remaining  beets  should  ba  ravnallng.  Beats,  other  then 
immediately  adjacent  baste,  that  are  significantly  eorrelnted  would  indicate 
dependency  within  the  response.  Such  dependency  would  be  preliminary  evidence 
for  either  the  operation  of  e  biological  control  mechanism  or  the  presence 
of  a  unitary  but  multlphaaic  response.  With  such  interpretation*  in  mind, 
it  would  be  of  interest  to  further  explore  the  dependency  etructure,  Vector 
analysla  might  provide  a  statistical  tool  for  this  purpose.  The  ability  to 
isolate  independent  factors  within  the  reaponae  is  appealing  conceptually 
although  practical  problems  of  tha  intsrprstation  of  factors  might  emerge. 

Other  practical  considerations  are  whether  or  not  to  rotate  the  factor 
matrix,  which  rotation  to  use,  and  whether  tha  factors  should  be  derived  from 
the  covariance  matrix  or  the  correlation  matrix.  (Tha  covariance  matrix  is  a 
posaibility  because  all  of  the  acorea  have  the  same  bpm  metric).  Even  the 
basic  correlation  matrix  should  answer,  however,  the  question  of  whether 
dependency  exists  with  the  cardiac  response  to  a  task. 

In  summary,  an  exploration  of  the  question  of  the  nature  of  the  response 
has  suggested  that  thia  problem  be  split  into  two  ssctlons.  1)  Can  the 
post-stimulation  response  be  predicted  from  the  pre-stimulation  variability? 

This  question  may  be  answerable  by  intercorrelatlng  intuitive  polnte  of 
Interest,  such  as  maxima  end  minima,  across  the  two  segment  of  tha  HR  response 
curve,  2)  What  are  the  dependency  characteristics  of  the  post-stimulation 
reaponae?  The  direction  suggested  to  answer  this  qusstlon  was  an  extensive 
correlational  analysis. 

Question  3s  Given  a  set  of  meaningful  components,  can  a  comparison 
across  tasks  be  made  of  these  components? 
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The  meaningful  comparlaon  oS  tha  cardiac  raaponaa  to  quadrant  placing 
and  coordinate  meigoty  la  the  essential  task  remaining.  Two  of  the  prevloualy 
euggeated  analysis  technlquea  can  be  uaed  to  show  a one  problem*  Involved  In  jtT 
auch  a  comparlaon.  •>;V 

In  dlacuaalng  thia  question,  the  reallatlc  aaeumptlon  will  be  made  that 
a  moderate  degree  of  dependency,  exists  within  the  raeponee  to  the  teak  and 
between  the  pre-period  and  reeponae  period.  If  in  fact  little  dependency 
exists,  then  relatively  atralght  forward  statistical  comparisons  are 
poaalble. 

One  approach  to  comparing  cardiac  reaponaaa  would  be  to  uca  the  reaulr.a 
of  the  factor  analyaia  suggested  earlier.  Hopefully,  thla  ana lye la  produced 
two  or  three  independent  factor*  that  were  interpretabla  in  terme  of  different 
influence*  on  cardiac  function.  If  one  of  thase  factor*  vae  interpretable  a a 
the  influence  of  the  paychologlcel  event,  than  primary  interest  would  be  in 
Identifying  this  factor  in  the  reeponae  to  both  taake  end  then  compering  the 
factor  loadings.  .It  la  likely,  however,  that  leea  clear  factors  would  emerge  ^ 
and  that  more  then  one  factor  would  aeam  Implicated  in  the  reeponae  to  the 
psychological  event.  In  either  cate,  tha  factor  structure  of  one  task  must 
ba  Interpreted  end  than  tha  factors  within  the  response  to  the  other  task 
muot  be  rotated  to  che  criterion  cf  matching  the  structure  of  the  first 
task  response.  Once  thia  la  achieved,  e  atatlatlcal  comparison  of  the  load¬ 
ing  on  each  factor  would  be  desirable,*  It  it  very  unclear  whether  some 
type  of  analysis  of  variance  on  factor  loadings  would  be  desirable  end 
justified,  (a. g.,  Factors  X  Tank  X  Order  X  Replications).  .Indeed  the  whole 
procedure  just  outlined  seems  auspect  and  suggests  that  other  forms  of 
multivariate  analysis  might  ba  preferable. 

The  use  of  Intuitive  peak*  and  valleys  is  another  way  of  comparing  teak 
responses.  These  scores  must  be  adjusted  in  some  way  by  prior  values,  how¬ 
ever.  This  necessity  seems  in  general  to  lead  to  a  proliferation  of  scores 
with  dubious  validities. 

-#• 

For  example,  a  response  maximum  score  may  be  corrected  by  covarying 
the  pre-response  maximum  and  minimum  and,  perhaps,  even  by  covarying  an 
immediately  preceding  response  minimum.  Seven  different  acoree  can  be 
generated  using  different  combinations  of  theaa  covariates.  These  eeven 
scores  plus  those  for  the  response  minima  could  be  analysed  one-by-one 
using  a  univariate  or  multivariate  analysis  of  variance.  Tha  high  degree 
of  lntra  response  score  dependency,  however,  poses  interpretive  problems. 

Similar  argument  can  be  made  for  and  against  a  bsat-for-beat  comparison 
of  two  response  curves. 


*  Or.  Robert  Chapman  also  of  the  Department  of  Experimental  Psychophysiology, 
WRAIR,  has  developed  ideas  along  these  lines  in  attempting  to  deal  with  the 
comparison  of  cortical  evoked  potentials.  His  ideas  suggested  this  applica¬ 
tion  to  heart  rate. 


30- 


In  conclusion,  th«  problem  of  how  Co  statistically  analyte  and 
characterise  a' complex,  dependent  ti«e-varyi,ig  response  he*  been  pdoed. 

The  quest ion  was  posed  in  terms  of  a  prototype  experiment  and  then  a 
specific  comparison  of  the  response-  .«*&aified'  the  question  in  more  detail.' 
Tenative  approaches  to  soms  aspect,*-  <jf  the  problem  were  of  feted  but  no 
truly  coherent  meads  of  characterising  and  analysing  the  data  could  ba 
of farad.  What  would  you  do? 
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ABSTRACT 


The  equations  of  chemical  kinetics  have  bean  applied  Co  combat  models  for 
over  half  a  century.  Because  of  wide  variability  in  military  performance  due 
to  personal  initiative,  terrain,  weather,  etc.,  such  applications  have  sometimes 
had  their  difficulties.  However,  kinetic  equations  are  very  useful  in  describing 
the  treatment  of  malignant  tumors  by  chemotherapy  and  radiotherapy.  A  fairly 
general  model  for  radiotherapy  la  proposed.  The  combat  analogy  is  that  of  an 
Invulnerable  blue  side  engaged  in  the  destruction  of  a  red  force  inextricably 
ititormeshed  with  a  defenseless  pro-blue  side,  e.g.,  an  in  the  use  of  air  power 
to  tiunpress  a  guerilla  insurrection.  It  ;la  demonstrated  how  the  model  answer.* 
aomt.  of  Che  conjectures  of  experimental  radiology.  Moreover,  the  model  shows 
liow  acuta  dose  external  oeam  treatment  may  be  used  to  simulate  continuous  low 
dose  therapy.  Till*  should  enable  (generally  preferred  whore  feasible)  "radium" 
treatments  to  be  employed  in  many  more  caeca  than  at  present. 


’’'This  research  was  supported  in  part  by  U.S.Armv  Contract;  DAAB09-71-R-0063 
and  by  Thu  National  Institutes  of  Health  Crane  0411430. 
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InttoAuctlon.  The  credit  for  introducing  differential  equation  models  into  the 
theory  of  combat  is  generally  given  to  Lanchester  [10].  Ilia  early  two  force 
models  are  of  the  general  form 

jjf  -  -aax  -  a#y  -  a3xy 
^  -  -bjX  -  bBy  -  b3xy  . 

Those  may  easily  be  extended  to  a  variety  of  heterogeneous  force  models. 

Below  wc  exhibit  a  combat  model  useful  in  formulating  strategies  in 
radiotherapy.  In  the  context  of  warfare,  it  is  a  description  of  the  use  of 
alrpower  in  a  friendly  area  occupied  by  an  enemy  force.  Lot  there  be  given  an 
enemy  with  strength  w  and  three  friendly  forces  with  strengths  x,  y,  and  z. 
Over  a  particular  interval  of  time,  let  the  equations  of  attrition  be  given  by 

dw 

dt  "  -C'W*  +  q«W 
^  •  -csxz  +  q„x 


di 

dt 


-c3yz  +  qyy 


dz 

dt 


0 


where  the  replacement  parameters  satisfy  qr  >  q„  >  q*  —  0  . 

Such  a  situation  might  arise  where  alrpower,  z,  la  to  be  used  in  destruc¬ 
tion  of  an  enemy  with  strength  w  .  Because  the  enemy  is  mixed  with  the  friendly 
inhabitants  of  the  area,  it  is  impossible  to  avoid  a  certain  amount  of  non-enemy 
injury  and  loss  of  life.  Some  of  this  attrition  will  occur  in  segments  of  the 
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population  -  a.g.,  village  elder*,  civil  resistance  forces ,  etc.  -  which  can  be 


satisfactorily  replaced  very  alowly.  These  are  the  force*  with  strength  x. 


Other  non-enemy  loss  of  life  will  include  persona  not  in  positions  essential  to 


the  resistance  effort.  However  ghastly  their  deaths  are  from  a  humanitarian 


standpoint,  their  niches  in  the  society  are  filled  relatively  swiftly.  These  are 


the  forces  with  strength  y. .  The  enemy  forces  are  considered  to  be  replaceable 


rather  slowly  -  a.g.,  by  Indoctrination  of  friendly  forces.  It  is  assumed  that 


a  fairly  constant  alrpower  committment  Is  made  by  the  friendly  aide  with  losses 


replaced  Immediately.  The  above  model  might  be  a  useful  description  of  e  number 


of  real  world  uses  of  airpdwer  -  e.g.,  some  of  the  bombing  campaign*  in  Imlo  ■ 


China.  The  problem  is  to  minimize  w  subject  to  constraints  on.  x  and  y 


It  happens  that  the  model  described  above  Is  also  useful  in  the  consider¬ 


ation  of  the  problem  of  obtaining  optimal  strategies  in  radiotherapy.  The 


effect  of  s  treatment  on  three  separate  cell  groupings  must  be  considered. 


First,  there  is  the  malignant  tissue  which  we  seek  to  destroy.  Next ,  there  uret 


connective,  vascular  and  nerve  tissues  which  are  replaced  so  slowly  that  they 


can  be  considered  virtually  irreparable.  Finally,  there  are  the  functional 


tissues,  skin,  muscle,  gut,  etc.  Undet  moderate  radiotherapy,  these  tissues 


are  replaced  very  swiftly  and  practically  need  not  be  considered  as  a  constraint 


in  the  treatment.  The  problem,  then,  can  be  treated  as  one  of  constrained 


minimization.  We  wish  to  destroy  as  much  neoplastic  tissue  as  possible  without 


causing  so  much  destruction  of  connective,  vascular  end  nerve  tissues  that  a 


necrosis  will  result. 


It  is  a  generally  accepted  view  that  low  rate  continuous  radiation  by 


radium  implants  is  preferred  to  high  rate  fractionated  external  beam  therapy 


[4,  7,  12,  13],  Unfortunately,  only  a  small  percentage  of  malignant  tumors  con 


be  treated  by  implantation. 
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A.  1.4 

Thfe  purpose  of  this  paper  is  to  uue  a  Lanchestpr  type  model  to  give  an 
Interrelation  between  continuous  and  fractionated  radiation  therapies.  Having 
achieved  this  objective,  we  shall  demonstrate  a  scheme  by  which  the  effect 
of  continuous  therapy  may  be  closely  approximated  by  a  fractionated  regime. 

£f*cu*aion.  All, our  modelling  has  been  carried  out  under  the  assumption*  of 
multi-target  theory  (for  an  excellent  introduction  to  the  auhject  see  Barendaen 

t  ,  ■ 

{1|).  Basically,  it  is  asaumed  that  each  cell  has  n  targets.  When  all  n  : 
targets  have  been  damaged,  Che  cell  1.  destroyed.'  A  convenient  neaoute  of  dose 
is.tho  number  of  D0 'a  delivered,  where  for  a  given  tj»pe  of  cell  a  dos:*  of  one 
D0  will  result  in  as  many  hits  a*.  ..target*  .t  All  targets  are  repaired  coh- 
currently  {5,  6,  20]  and  at  a  ra lie  which  is  a  parameter  of  the  model.  The  ' 
equetion  for  the  surviving  fraction  of  calls  at  thp  end  of  a  high  rate  treat- 

.  li  , 

ment  of  dose  D  (in  D0 '*)  is  given  by  .i 

.*  .  ..  -D.n  ■< 

1  i  • 


|  The  change  of  sensitivity  during  the  cycle  12,  8,  11,  15,  16,  17,  19]  is 

accounted  for  by  a  variation  in  the  target  number  [15].  This  variation  is  approx 
imated  by  breaking  the  cycle  into  two  phases,  one  a  resistant  phase  irl,  which 

,  cells  have  a  high  target  number,  the  other  a  sensitive  phase  in  which  a  single 

■:  hit  kill*  the  cell, 

I  •  - 

;  In  Figure  1  we  show  our  basic  model.  In  a  short  interval  of  time,  dt,  a 

proportion  k,  dt  of  the  cells  is  leave  resistant  phase.  During  the  same 
'  timc  interval»  a  proportion  k3  dt  leave  the  sensitive  phase.  During  mitosis 

each  cell  Is  replaced  by  an  average  of  q  daughter  cells  which  immediately 
enter  the  sensitive  phase.  When  no  cells  die,  q  is  2.  0  .  For  nonprolifer- 
atinp,  tissue,  q  is  1.0.  Generally,  we  have  assumed  a  a  value  nf  i.i 
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It  In  assumed  that  during  continuous  radiotherapy,  in  addition  to  single 
target  killing,  there  is  an  appreciable  amount  of  single  hit  killing  [1,  3,  9, 
14,  19) .  This  damage  may  result  from  high  LET  traversal  [3,  14,  1.9] ,  the' 
accumulation  of  a  few  relatively  hish  energy  traversals,  or  some  target  which 

■  "  “l*  •  .  1  " 

cannot  be  repaired  but  which  has  a  higher  D0  than  the  reparable  ones.  In 
systems  with  an  appreciable  number  of  targets,  alnoat  #11  killing  frera  low  rate 

.<  -•  f  ,.il  ' 

radiation  appears  to  be  due  tp  irreparable  damage.  1  ' 

In  continuous  radiotherapy,  killing  is  achieved  at  rates  ka  and  k4 
from  the  sensitive  and  resistant  phases  respectively.  Hence,  i  k3  is'  simply  •<;  r 
the  doac  rate  (in  units  of  Dc ) .  Assuming  that #  fixed  fraction  u>  of  the  j.’ 
dose  delivered  does  irreparable  Jamage,  k4  Is  w  times  the  dose  rath'.  ,!i: 

To  be  more  specific,  let 

I*  f;- 

,  "  l!i  ■ 

x'*(t)  •*  the  number  of  cells  in  the  resistant  phase  at  time  t  ,. 

y'v(t)  »  tin:  number  of  cells  in  the  sensitive  phase  at  time  t  . 

If  we  normalize  these  quantities  (in  order  to  talk  about  surviving  proportions), 
we  have 

x(t)  -  x*(t)/(x*(0)  +  y*(0>) 

y(t)  -  y*(t)/(x*(0)  +  y*(0))  . 

Then  the  kinetic  equations  for  continuous  radiation  are  given  by 

~  -  -(kj  +  k4)x  +  k3qy  , 

dt  “  k,x  -(kB  +  k3)y  . 

The  solution  is  given  by 
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x(t)  -  flj  exp[  (-ct  +  c„)t]  +  a„  exp[-(cx  +  c„)t]  ,, 
y(t)  »  bx  expK-Cj  +  ca)t]  +  b8  expf-(cl  +  c8).t]  , 
where  ■  . 

ba  ■  frt  +  K  -  ^  ‘  cB^<*2^kaca  f  -  ^ 

t  ! 

T  “  cell  cycle  time 

bi  “  kgr  *  ba 

.  qk., _ 

1  k  i  +  k4  -  C  j  +  c”  1 

■■  “  kjr  ‘  •*» 

k,  +  kc  +  k,  +  1;. 

C1  “  — -  2 - - - 

cB  "  fc\  -  kjk3  -  k1kri;i-q)  -  k4  <k9  +  k„> 

At  the  end  of  the  treatment  (t  ■  t, ) ,  the  fraction  of  surviving  cells  is 
given  by 

f  «  x(t# }  -t*  y(t,)  . 

To  obtain  an  algorithm  for  fractionated  radiation,  we  use  the  above 
formulae  during  the  time  between  treatments  with  k3  “  k4  ■  0,  The  surviving 
proportion  at  the  end  of  a  fraction  is  given  by 

x (t  +  0)  «  x(t)(c_uiD(l  -  (1  -  e‘(1'u,)£,)»)] 

y(t  +  0)  «  y(t)o*D  . 
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Below,  we  alter  the  parameters  In  the  model  one  at  a  time  holding  the 
other  parameters  at  their  "standard  conditions"  values  (see  Fig.  1).  Aa  previously 
noted,  in  a  course  of  fractionated  treatment  the  target  number  In  the  resistant 
phase  is  quite  important.  This  fact  can  be  seen  from  Figure  2.  The  surviving 
fraction  increases  dramatically  as  tha  number  of  targets  increases  from  one  to  < 
six.  It  then  Increases  at  a  more  leisurely  rate  to  an  asymptotic  value  where 

•  •'  ’  i  i 

the  only  killing  is  of  the  single  target  variety  in  both  the  sensitive  ahd 
resistant  phases. 

In  Figures  3  and  4  we  show  the  effect  of  the  percent  of  the  cell  cycle 
spent  in  the  sensitive  phase  on  cell  survival  for  both  mpdoa  of  radlatibn 
exposure.  In  both  caseB,  the  surviving  fraction  decreases  with  increasing  time: 
in  the  sensitive  phase  -  more  dramatically  at  low  dose  rates.  In  the  fractionated 
course,  higher  dose  rates  always  give  lower  surviving  fractions.  This  is  not  so 
for  continuous  treatment.  At  some  point,  the  effect  reverses  and  low  rates  give 
a  lower  survival  fraction  than  the  high  rates.  The  greater  the  proportion  cf 
the  time  in  the  sensitive  phase  the  greater  this  effect,  so  that  with  forty 
percent  in  the  sensitive  phase  the  dose  given  at  5  Dp 'b  per  cycle  produces  » 

surviving  fraction  one-hundred  times  lower  than  when  given  at  25  Do 's  per  cycle. 
This  might  seem  to  he  an  argument  in  favor  of  dose  rates  lower  than  the  20  -  23 
D0's  per  cycle  given  in  intracaviary  therapy.  It  is  doubtful,  however,  that  the 
proportion  of  time  in  the  sensitive  phase  approaches  forty  percent  for  any 
neoplastic  tissues.  Moreover,  the  points  of  crossover  of  the  lines  move  to  the 
right  with  increasing  q  .  A  gross  lessening  of  dose  rate  could  reduce  the 
treatment  effectiveness  due  to  an  increase  in  q  during  the  period  of  treatment. 
Nevertheless,  moderately  lower  doae  rates  than  currently  applied  in  radium 
therapy  might  very  well  be  desirable  in  some  cases. 

The  possibility  of  the  production  of  lower  survival  fractions  by  low  dose 
rates  is,  of  course,  due  to  redistribution,  in  the  fractionated  course,  the 
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Interval  between  treatments  la  half  of  a  cycle,  and  redistribution  of  surviving 
cells  into  a  mixture  of  sensitive  and  resistant  phases  is  largely  completed  during 
this  time.  In  a  course  of  continuous  radiation  there  is  a  different  effect  ea 
ahown  in  Figure  5.  The  fraction  of  cells  In  the  sensitive  phase  fells  rapidly 
early  in  treatment  but  quickly  aymptotes .  The  value  of  the  asymptote  depends 
to  a  large  extent  on  the  rate  at  which  cells  in  the  compartment  are. killed. 

Hence,  there  is  a  greater  fraction  of  cells  in  the  sersiciv.’  phase  at  low  dose 
rates.  Consequently,  less  of  the  dose  is  wasted  on  cells  in  the  resistant  phase. 

In  Figures  6  and  7  we  exhibit  the  effect  of  the  proliferation  rate,  q, 
for  continuous  and  fractionated  strategies.  A  very  important  point  which  is 
observed  is  the  fact  that  the  efficacy  of  continuous  treatment  is  little  changed 
by  oven  a  drastic  change  in  q  .  We  conjecture  that  this  may  be  an  important 
factor  in  the  generally  preferred  result!  obtained  by  radium  therapy.  In  the 
fractionated  course  the  surviving  fraction  changes  by  at  least  a  factor  of  100 
as  q  goes  from  one  to  two. 

In  Figures  8  and  9  we  examine  the  effect  of  the.  proportion  of  single  hit 
events  in  the  resistant  phase.  In  the  case  of  continuous  radiation  there  is  a 
great  change  in  surviving  fractions  with  uj  .  In  the  fractionated  case,  the 
change  is  much  less.  V 

V. 

< 

In  Figure  10  we  see  the  effect  of  the  number  of  high  dose  rate  treatments 
per  cell  cycle  on  the  surviving  fraction  of  cells.  The  change  in  surviving 
fraction  is  large  as  wc  go  from  one  to  three  treatments  per  cycle.  However, 

t 

for  the  dose  rates  considered  there  is  essentially  no  change  in  surviving 
fraction  with  change  in  the  number  of  courses  per  cycle  if  this  number  is  at 
least  seven,  lieyond  this  number,  only  single  target  killing  takas  place. 

Figures  11  and  12  Eive  the  surviving  fraction  as  a  function  of  the  total 
dose  delivered.  There  is  little  dose  rate  effect  in  the  continuous  case, 
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although  the  surviving  fraction  changes  a  bit  more  rapidly  with  total  dose  at 
low  dose  rates.  However,  lr.  the  fractionated  case,  the  dose  rate  effect  1b 
significant,  since  multltargct  killing  Increases  with  the  dose  rate. 

In  Figure  13  we  note  the  lowering  in  the  killing  ability  of  fractionated 
radiation  if  the  target  number  in  the  resistant  phase  is  increaded  to  10. 

Let  us  now  consider  a  practical  example  to  interrelate  confc'lnupiu  and 
fractionated  treatments.  It  shall  be  assumed  thst  Dp  for  both  normal  and 
neoplastic  tissues  is  150  with  a  cell  cycle  time  of  48  hours.  It  Is  further 
assumed  that  **  .3,  q  *  1.1,  the  proportion  ol‘  time  in  the  sensitive  phase  is 
.2,  and  the  target  nustocr  in  the  resistant  phase  Is  6.  The  total  dose  of 
radiation  is  limited  by  the  allowable  amount  of  killing  of  slowly  cycling 
tissues.  A  typical  continuous  treatment  consists  of  s  total  dote  of  7000  rada 
over  a  period  of  7  days  by  radium  implants.  This  corresponds  to  13.3  D0's  per 
cycle  and  a  total  do3e  of  Uf>.  7  Dfi 'a  .  The  surviving  fraction  of  the  slowly 
cycling  connective,  vascular  and  nerve  tissue*  is  given  by 

f  -  -  e-14  ~  10"s  . 

The  correponding  surviving  fraction  of  neoplastic  tissue  is  obtained  from 
Figure  11  as  slightly  less  than  10“7(,44  X  10“7). 

Mow  let  us  consider  a  fractionated  treatment  of  the  same  tumor  with  a 
regime  using  art  acute  dose  of  300  rads  (4  D0 's)  per  cell  cycle.  We  wish  to 
determine  the  total  dose,  which  would  result  in  the  same  degree  of  destruction  of 
slowly  cycling  tissue  as  the  radium  stategy  given  above.  This  is  easily  obtained 
by  solving 

f c-( ,3>3[ 1  .  (1  .  c-(.7)3)0j]B  „  e~14  . 

We  obtain  a  solution  of  17.4  treatments,  or  34,8  D;, 's  (5200  rads).  Using 
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11 

Figure  12.  we  find  a  surviving  fraction  of  malignant  tissue  slightly  in  excess 

I 

of  10‘*  (1.09  X  l<r?).v'- 

In  this  example  the  model  gives  neoplastic  survival  fractions  for  contin¬ 
uous  and  acute  fractionated  dose  therapies  which  differ  somewhat  less  than  , a  factor 
of  3  in  favor  of  the  continuous  strategies^  It  should  be  remembered  that  a 
constant  value  of  q  «  1.1  has  been  assumed  throughout.  Should  q  increase 
beyond  this  value  during  the  course  of  treatment ,  then  as  Figures  3  end  4 
demonstrate,  the  relative  effectiveness  of  the  continuous  strategy  will  increase. 

Zn  general,  the  relative  constancy  of  the  killing  ability  of 
continuous  radiotherapy  with  changes  in  the  cell  kinetics  of  the  neoplastic 
tissue  is  a  decided  advantage  over  the  usual  fractionated  stretaglee. 

1  Inasmuch  as  radium  implants  can  be  employed  in  only  a  small  fraction  of 

f 

■  tumors,  it  shall  now  be  our  teak  to  demonstrate  how  the  proponed  model  may  be 

used  to  devise  approximatlone  to  continuous  treatment  by  fractionated  external 

f 

*  beam  therapy,  (The  approximation  of  a  constant  valued  function  by  a  Dirac  comb 

is, of  course,  a  common  technique  in  a  number  of  fields,  e.g,,  the  design  of 
filters  In  time  series  analysis.)  For  approximating  the  radium  implant  strategy 
above,  it  might  be  undesirable  to  deliver  1000  rads  per  day  in  small  incremental 
acute  doses ,  since  this  would  involve,  with  present  facilities,  a  constant 
wheeling  of  the  patient  in  and  out  of  the  treatmant  room.  (Perhaps  In  the 
future  there  will  be  a  floor  in  major  hospitals  devoted  to  rooms  for  radio¬ 
therapy  patients  with  cobalt  machines,  etc.,  located  on  tracks  on  the  floor 
above  so  that  it  is  possible  to  treat  the  patient  in  his  own  room.)  However, 
delivering  1000  rads  evexy  two  days  in  125  red  doses  every  6  hours  should  be 
feasible.  This  would  be  6.65  Dq's  per  cell  cycle  instead  of  13.30,  An  exam¬ 
ination  of  Figure  11  shows  that  under  the  cell  kinetic  conditions  listed,  a 
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continuous  strategy  using  6.65  D, 's  per  cell  cycla  would  actually  increase  some- 
what  the  total  killing  of  neoplastic  tissue  for  a  total  dose  of  46.7  Dp's  (7000 
rads).  Figure  10  shows  that  at  a  doss  of  6.65  D0's  par  call  cycla,  8  equal 
treatments  par  call  cycle  give  a  slightly  smaller  surviving  fraction  than  an 

infinite  number  of  treatments  (i.e.,  continuous  therapy)  par  cycle.  Hanca# 

•  . .  **“  .  .  .  . .  . .  “  " 

Figure  11  may  be  used  to  give  conservative  estimates  of  the  surviving  fraction 
of  malignant  tissue.  However,  we  must  still  determine  the  total  dose  allowable 
under  this  regime  to  kill  no  more  noncycling  tissue  than  in  the  7000  rad  radium 
strategy.  Solving  for  m  in 

(axpl  - . 3 <  125 ) / 150]  (1  -  exp{-.7<125/150))J°  )ro  -  e‘l4  , 

we  obtain  a  valua  of  m  ■  56.  Hence  a  total  dose  of  125  x  56  ■  7000  rads  is 
allowable.  Using  Figure  11,  we  obtain  a  conservative  value  of  .44  x  l<f7  ae  the 
surviving  fraction  of  malignant  tissue.  Thus  with  the  exception  of  better 
control  of  dose  distribution  in  the  tumor  by  actual  implantation,  it  would  appear 
that  the  effect  of  continuous  treatment  via  radium  implants  may  be  nicely  approxi¬ 
mated  by  acute  dose  fractionated  treatment. 

\ 

Conclusions.  We  have  seen  that  a  Lanchester  type  combat  model  gives  a  straight¬ 
forward  maans  for  interrelating  continuous  and  fractionated  strategies  of 
radiotherapy.  At  this  stage  In  the  modeling  we  have  not  taken  account  of  the 
dynamics  of  the  modal  parameters  during  Che  courae  of  treatment.  Undoubtedly, 
the  length  of  the  cell  cycles,  the  proportion  of  time  in  the  sensitive  phase  and 
the  value  of  q  change  during  the  treatment.  In  addition,  the  pooling  of  all 
cella  with  target  numbers  other  than  unity  into  a  compartment  with  one  target 
number  la  an  oversimplification.  Moreover,  the  possibility  of  a  growth  fraction 
of  leas  than  unity  has  not  been  considered.  A  mixture  of  cells,  some  stopped  in 
the  resistant  phase  of  the  model  and  some  cycling,  can  be  handled.  However, 
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pilot  calculation*  indicated  that  if  the  growth  fraction  stay*  clot a  to  unity 
(say  at  laaat  .9)  tha  surviving  fraction  at  triatoent  and  la  not  much  affegfjftf. 

We  with  to  raltarata  that  calculations  using  nodali  of  solely  multi* 
target  killing  by  continuous  low  rate  radiation  predict’  much  leas  destruction  , 
then  ie  observed.  In  s  fashion  similar  to  others,  we  hive  postulated  theta 
fraction  of  the  dose  delivered  does  single  target  killing  regardless  of  the 
number  of  in  the  cell.  It  is  not  surprising  that  such  a  factor  Would 

have  gone  largely  unnoticed,  since  it  makes  comparatively  little  difference  in 
a  fractionated  acute  does  regime.  Contrary  to  ona's  initial  reaction  when 
first  exploring  the  necessity  for  positive  iu  ,  it  is  not  a  bonus  which  makes" 
continuous  low  rate  radiotherapy  feasible.  One  hit  LET  killing  limits  the 
flexibility  of  treatment,  since  without  It  one  could  much  more  fully  exploit 
the  differences  between  the  kinetic  parameters  of  neoplastic  and  normal  tissues 
by  adjusting  dose  rate  and  schedule. 

The  principle  reeson  for  leek  of  consideration  of  these  matters  is  the 
fact  that  changes, in  cellular  dynamics  during  treatment  seem  to  be  poorly  under* 
stood.  Moreover,  it  seems  reasonable  to  study  the  simpler  static  parameter 
case  before  proceeding  to  the  more  complex  dynamic  parameter  case. 

The  model  gives  a  plausible  explanation  for  the  fact  that  continuous 
radiotherapy  has  generally  produced  bettsr  results  than  fractionated  acute  dose 
treatment.  Moreover,  it  gives  a  straightforward  means  for  obtaining  radium- 
like  results  by  a  regime  of  closely  spaced  fractionated  treatments.  Thus 
the  advantages  of  radium  treatment  should  be  available  to  a  much  greater  number 
of  patients  than  is  presently  tha  case. 
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Figure  ). 


Resistant 

_ 

Sensitive 

J1* 

STANDARD  CONDITIONS: 

Treatment  conditions  - 

Course  of  treatment  is  45  Df !a 

Proportion  of  radiation  which  does  single  hit  killing  is  0.3 
In  fractionated  high  rate  treatment,  two  fractions  pet  cell  cycle  are 
given 

Kinetic  conditions  - 

Proportion  of  time  in  sensitive  phase  is  0.2  of  the  cycle 
Proportion  of  time  in  resistant  phase  is  0.8  of  the  cycle 

The  proliferation  factor  q  is  1.1 
The  target  number  is  6 


A.  1.15 

Figure  2 

NUMBER  OF  T  ARC  El'S  va  LOG  SURVIVORS 

("TWO  FRACTIONS  PER  CYCLE) 

0)  *  0.3 

Percent  o£  cycle  in  sensitive  phase  -  0.2 
Total  treatment  dose  >'45  De 

’  ■  *•* .  . . .  ;j . 

Dose  ratio  top  to  bottom  1,  5,  (1)  D0  per  cycle. 


g 


Target  Number 
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Proportion 


Kigura  5 


Percent  of  cycle  In  sensitive  phase  «  .2 

Target  number  -  6 

Total  treatment  doae  ■  45  D0 

Doae  rate  bottom  to  top  left  5,  25,  (5) 
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Figure  7 

PROLIFERATION  RATE  vs  LOG  SURVIVORS 

(TWO  FRACTIONS  PER  CYCLE) 

u>  ■  .3 

Percent;  of  cycle  in  sensitive  phase1  ■  .2 

Target  number  -  6 

Total  treatment  dose  n  45  De 

Dose  rate  (top  to  bottom)  1,  5,  (1) 
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EFFECT  OF  u)  ON  TREATMENT  (CONTINUOUS  RADIATION) 
Total  treatment  dose  ■  45  D0 
Proportion  of  cycle  In  sensitive  phase  ■  0.2 
Target  number  -  6 


Lcg10  Surviving  Fraction 


Figure  9 


tt> V*  LOG  SURVIVING  PROPORTION  (FRACTIONATED) 
Dose  rate  top  to  bottom  I,  5,  (1) 

Target  number  >6 

Proportion  of  cycle  In  sensitive  phase  ■  ,2 
Total  treatment  dose  -  45  D0 
q  »  1.1 


8 
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iw  x  100 
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Logl0  Surviving  Fraction 
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Figure  10 

NUMBER  OF  FRACTIONS  PER  CYCLE  v»  LOG  PROPORTION  SURVIVING 
a>  ■  0.3 

Target  number  -  6 

Proportion  of  cycle  in  aeneitive  phase  -  .2 
Total  treatment  dose  ■  45  D0 
q  -  1.1 

Dose  rate  top  to  bottom  left  1,  0,  (2) 
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Surviving  Fraction 
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Figure  13 


EFFECT  OF  TOT£L  DOSE  ON  TREATMENT  (FRACTIONATED) 

TWO  FRACTIONS  PER  CELL  CYCLE 

<D  ■  0,3 

Proportion  of  cycle  in  aeneitive  pheae  ■  .2 
Target  number  ■  10 
q  ■  1.1 

Dote  rate  1,  5,  (1)  De  per  cycle 


g 


Total  Treatment  Doso  (in  D0  unite) 
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A  COMPUTER  PROGRAM  FOR  TRXCHOTOMOUS  BIOASSAY 

Clifford  J.  Maloney 
Fred  3,  Yaaada 

National  Institutes  of  Health 

A  statistical  technique  is  applied  so  frequently  in  a  biological 
context  that  it  la  widely  described  as  bloassay,  though  it  is  by  no  means 
confined  to  such  applications  and  applications  in  other  fields,  e.g., 
explosive  sensitivity  or  armor  penetrability,1  are  common,  The  standard 
text  is  that  of  Finney,2  though  here  again  the  name,  problt  analysis, 
applies  only  to  one  special  approach  and  the  book  in  fact  discusses 
other  approaches.  It  is  unfortunate  that  the  terms  "quantal  regression" 
or  "quantal  analysis,"  which  are  neutral,  descriptive,  and  not  preempted 
for  other  purposes  ere  not  more  widely  adopted. 

A  second  characteristic  of  the  great  majority  of  contributiohe  to 
bioassay  (quantal  regression)  is  that  the  response  variable  la  assumed 
to  take  juat  one  of  two  values,  response  (which  la  often  death  or  compo¬ 
nent  failure)  end  non-responae.  In  tome  instances,  however,  the  reeponae 
may  take  one  of  several  forma  which  can  be  ordered  aa  to  severity.  In  the 
case  of  an  insect,  for  example,  the  response  could  be  moribund  as  wall 
as  dead.  In  the  armor  penetrability  example,  the  response  could  be  partial 
aa  well  as  complete  penetration.  In  such  cares  a  method  of  analysis  that 
takes  both  types  of  response  into  consideration  should  give  more  precise 
answers  than  the  device  that  lumps  intermediate  responses  either  with  the 
responders  or  with  the  non-responders  to  permit  application  of  standard 
analysis. 

A  solution  for  this  situation  was  supplied  some  years  ago  by 
Burland  et.  al.,3  giving  explicit  equations  for  the  case  of  three  classes 
of  response,  the  trlchotomous  case;  though  an  appendix  treats  the  general 
situation  of  S  outcome  classes  of  one  relation.  The  fully  worked  out 

1  Golub,  Abraham,  and  Grubba,  Frank.  "Analysis  of  Sensitivity  Experiments 
When  the  levels  of  Stimulus  Cannot  be  Controlled." 

JASA,  June  1956,  Vol.  51,  pp.  257-265. 

2  Finney,  D.  J.  PROBIT  ANALYSIS,  Third  Edition,  Cambridge  University,  Press,  1971 

3  Gurland,  John,  et,  al.  "Polychotomoua  Quantal  Response  in  Biological  Assay," 
Biometrics,  Vol,  16,  No.  3,  September  1960,  pp.  382-398. 
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computing  formula*  for  ralativa  potancy — including  data  for  both  a  taat 
product  and  a  anandard,  aach  with  two  ordarad  claacaa  of  outcoMa~gg|m 
appllad  to  an  illuatrativa  example,  Tha  particular  example  happana  to 
hava  tha  aamd  number  of  aubjacta  exposed  at  all  doaaa  on  on  relation  and 
for  both  relation*— taat  and  atandard— though  thla  la  not  Monetary,**  It 
way  be  helpful  to  aoae  reader*  of  the  Ourland  Papa;  to  be  told  that  doaea 
given  in  Table  1  are  to  b*  converted  to  common  logarlthaa,  though  tha 
paper  itaalf  aakaa  no  Motion  of  thla  step.  'S 

Thla  paper  raporta  the  fact  that  the  appropriate  calculatlona  have 
baan  programmed  In  tha  BASIC  and  FORTRAN  language*  and  appllad  to  tha 
axaapla  in  Gurland'a  paper.  Ganarai  availability  of  both  prograao  It  con¬ 
templated.  Theaa  flrat  program*  are  limited  to  tha  apadfic  one  cycle 
calculation  daacribed  In  (3) .  Modification  to  parait  Iterative  fitting 
la  modeat,  Flnnay  appllad  hla  generalised  computer  program  to  thla  caaa 
also,4 5  but  hla  program  la  not  directly  applicable  to  currently  available 
computer* . 


4  Gurland,  John.  Paraonal  Communication. 

5  Ibid,  page*  223-226. 
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CRITERIA  FOR  A  BIOCELLULAR  MODEL 


George  z.  Uvin 
Vulnerability  Laboratory 
DEL,  ARDC 

Abardaan  Proving  Ground,  Maryland 


Wa  ara  concarnad  with  tha  design  of  a  biocallular  nodal.  Thin 
nodal  la  to  bp  uaad  to  evaluate  tha  modification  of  performance  of  an 
anlnal  ayatan  consequent  to  tha  abaorptlon  of  energy  by  tha  anlnal. 

Pravloualy  reported  work  haa  had  to  do  with  tha  application  of 
apaclflc,  non-daatructlva  analytical  procaduras  for  tha  aaaaaanant  of 
traunatlc  conaaquancaa,  (ARO-D  Report  69-2) 

A  conaidarabla  effort  haa  baan  expanded  on  animal  nodala.  However, 
littla  attantlon  haa  baan  given  to  tha  phyalologlcal  aapacta  of  tha  problem. 

Moat  of  tha  publications  have  had  to  do  with  a  mathematical  type  of  analyala. 
Thla,  I  think,  will  ba  of  value  whan  wa  have  learned  more  about  the  bio¬ 
chemistry,  biophysics  and  bionics  of  tha  systems  involved. 

It  is  auggaatad  that  tha  criteria  Include i 

Tha  ralationahip  between  tha  biochemical  composition  and  tha 
•pacific  functionality  of  tha  ayatam(s)  under  consideration!  Protalna, 

Nucleic  Acids,  Nuclao-proteina,  Polysaccharides,  Lipoida,  Lecithins,  Caphallns... 
Tha  mechanism  of  anargy  absorption  by  tha  system!  Tha  origin  of 

spectra. 

The  aqulpartation  of  tha  absorbed  energy!  Formation  of  wound  tracts. 
Tha  Initiation  of  atom  and  free  radical  reaction  chains . 

Tha  dimensional-physiological  nature  of  tha  systems!  Soft  tissue, 

Hard  tlaaua,  Heart,  Kidney,  Brain,  Lungs,  Muscle,  Nerve,  Circulatory 
systems.... 

Feedback  effects!  Secondard  traumas.  Shock, 


SUMMARY 

An  attempt  la  being  made  to  obtain  a  perspective  of  the  reality 
of  animal-teak  performance  In  terms  which  are  involved  in  tha  performance. 
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FACTORIAL  EXPERIMENTS  OP  SMALL  ASMS  WEAPON  FIRE  CONTROL 
Adolph  P.  Kavalec 

Quality  Assurance  Dirac tot at* ,  Frank ford  Arsenal, 
Philadalphia,  Pannaylvanla 


Frankford  Araanal,  in  Philadalphia,  ia  engaged  ia  tha  raeaarch 
and  development  of  fira  control  inatruaants  for  a  wide  variety  of 
uaaa  auch  aa  eights,  rangaflndera  and  othar  inatruaanta  for  hall- 
cop  tar  a,  hovitaara,  tanka,  artlllary,  sort are,  racolllaaa  rltUl,  aa 
wall  aa  vahida  diagnostic  equipment  and  computers.  Thin  papar  daala 
with  tha  taat  and  analyaia  of  email  arwa  waapona  firad  from  moving 
trackad  vahiclaa. 

Futura  track  vehicle  paraonnal  carriara  are  being  designed 
with  porta  to  enable  troopa  lnaida  tha  vehicle  to  fire  hand  held 
weapons.  Instability  of  tha  weapon  caused  by  vehicle  motion  can  be 
asperated  into  twe  categories I 

1.  Lateral  shifts  in  line  of  sight  to  othar  parallel 
Unas  of  sight. 

2.  Change  in  direction  of  the  lina  of  alglit. 

Lateral  vibration  can  change  the  point  of  impact  on  tha  target 
by  only  the  dlstanca  over  which  the  weapon  moves,  If  the  muzsle  and 
butt  are  moved  10  millimeters  in  tha  same  direction,  impact  will  be 
10  millimeters  from  where  It  would  have  impacted  in  the  original  posi¬ 
tion.  If  the  muzzle  move*  10  millimeters  and  the  butt  of  the  weapon 
remains  stationary,  the  impact  at  a  target  at  100  meters  would  be  1 
meter  from  the  point  at  which  the  original  line  of  sight  would  have 
caused  it  to  impact. 

Thla  is  the  same  change  of  angle  characteristic  of  vibration 
which  causes  blur  In  cameras  and  optical  viewing  devices.  Gyros,  whose 
rotating  maas  directly  reslats  motions  which  would  cause  a  change  in 
angle,  have  been  used  successfully  in  aerial  photography  for  some  time 
and  have  proven  effective  in  reducing  blur  in  military  binoculars  when 
used  In  moving  track  vehicles  as  recent  teats  performed  by  Frankford 
Arsenal  have  demonstrated. 

A  proposal  was  made  by  Frankford  Arsenal  to  the  US  Army  Small 
Arms  Systems  Agency  in  1970  to  perform  a  limited  field  evaluation  of 
gyro  stabilized  weapons.  The  primary  objective 


The  rest  of  this  article  wee  reproduced  photographically  from  the  manu¬ 
script  submitted  by  the  author. 
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was  to  da tannins  the  effect  on  hit  probability  of  a  weapon 
with  standard  sights  and  a  stabiliser  whan  compared  to  the 
sene  weapon  without  stabilisation  from  moving  vehicles. 
Secondary  objectives  were  to  determine  .if  a  reflex  sight 
offered  any  Improvement  in  hit  probability  with  and  without 
stabilisation  from  moving  vshlclas. 

A  Kenyan  K8-6A  stablllier,  which  can  be  attached 
externally  to  the  weapon,  was  ths  stabiliser  utilised 
for  this  test.  Ths  Kenyan  KS-6A  was  previously  selected 
from  various  other  stabilisation  devices  as  having  those 
charscteriatics  which  make  it  Ideal  for  une  with  email 
arms  weapons.  Easily  attachable,  small  site  power  supply, 
etc.  The  stabiliser  utilizes  two  encapsulated  precision-  *  - 
i  balanced  gyros  mounted  In  spring  restrained  gimbals  which 

l  resist  pitch  and  yaw  motion.  Tha  internal  operation  of 

?'  the  gyro  consists  of  heavy  mass  wheels  which  are  rotated  by 

400  cycle,  155  volt  AC  motors  at  20  to  22,000  RPM.  The 
stabilizer  was  attached  to  the  pistol  grip  of  the  M16 
Rifle.  The  reflex  sight  used  was  a  commercially  available 
reflex  sight  with  an  electrically  illuminated  reticle  dot. 

The  dot  le  projected  to  the  plane  of  tha  target  and  reflected 
to  the  eye  by  the  front  surface  of  the  winduw  through  which 
the  target  Is  viewed.  The  reflex  sight  was  attached  to  the 
carrying  handle  of  the  rifle  ani  offers  a  simplification  of 
the  sighting  task  by  reducing  the  number  of  points  to  be 
aligned  for  aiming  from  4  to  2.  Aiming  is  accomplished  by 
simply  placing  the  dot  of  light  on' the  target.  The  weapon 
sight  requires  the  alignment  of  tha  front  eight,  rear  sight, 
and  eye  with  the  target,  whereas  the  reflex  eight  requires 
that  only  the  lighted  dot  reticle  be  aligned  with  the  target. 


TEgPjSStfN 

Normally,  the  type  of  test  design  is  dictated  by  the 
objectives  of  the  test,  the  Information  sought  and  the  cost 
considerations  associated  with  the  degree  of  precision,  sample 
size  and  conduct  of  tha  test.  Hie  initial  approach  was  to 
consider  a  tingle  factor  approach  to  the  tasting  of  small 
arms  fire  control.  The  single  factor  approach  waa  rejected 
since  it  was  likely  to  provide  a  number  cf  disconnected 
pieces  of  information  that  would  be  difficult  to  piece 
together  to  obtain  conclusive  results.  For  example,  in  order 
to  perform  an  experiment  on  a  two-level  factor  (weapon  sight. 
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reflex  eight) ,  some  decisions  rouit  be  made  about  the  levels 
of  other  factors:  Speed,  stabilization,  shooters  (shall  '  > 
the  fire  control  be  studied  at  8,  10,  or  121  MPH  in  a  moving 
vehicle  stabilized  with  shooter  1,  2,  3  or  4?).  The 
experiment  would  reveal  the  effects  of  the  two- level  factor, 
for  this  particular  combination  of  speed,  stabilization  end 
shooter.  . 

It  was  not  known  whether  or  not  the  factors  were  ’ 
Independent  and  since  there  were  several  levels  of  each 
factor,  a  factorial  approach  was  taken  cp  examine  the  effects 
on  hit  probability  of  the  weapon  eights  and  reflex  eight  at 
varloue  speed*,  stabilization  and  with  zaveral  Shooters. 

The'  factorial  approach  examines  all  the  main  factors  and 
their  interactions  simultaneously  and  would  provide  the 
maximum  amount  of  Information  to  be  returned.  The  tests  of 
hypotheses  are:  ' 

1.  Each  factor  has  no  effect  on  the  overall  mean. 

1  2.  There  Is  no  interaction  between  factors.  , 

This  factorial  approach  provides  for  the  effects  of  the 
two-level  factor  with  any  combination  of  speed,  stabilization 
and  shooter  that  is  iiicltoded  In  the  experiment  and  it  was 
expected  that  inferences  could  be  made  easily  with  this 
design. 

The  test  was  divided  into  two  phases: 

1.  Stationary  firing. 

2.  Moving  vehicle  firing. 

Tho  test  matrix  for  stationary  firing  measured  tho  mean 
radius  of  10  round  shot  groups.  This  was  a  4x3x2x2  factorial 
planned  to  be  replicated  four  times.  Similar  matrices  were 
also  set  up  for  the  moving  vehicle  exercise.  Tho  moving 
vehicle  wa.*:  separated  into  a  "toward"  target  and  "acrose" 
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target  exercise.  The  data  collected  for  the  moving  vehicle 
firing  was  the  number  of  hits  in  each  10  round  firing.  This 
also  was  planned  to  be  replicated  four  times. 

Originally,  each  replication  would  be  entered  as  a 
data  point  in  the  analysis,  and  since  there  would  be  equal 
subclass  numbers  (sample  size),  the  computer  program  was 
written  with  this  in  mind  before  the  completion  of  the  test. 
After  the  testing  was  completed,  several  cases  had  more  than 
four  replications  and  in  other  cases  less  than  four  replica¬ 
tions.  Hie  methods  available  of  "filling  in"  missing  data 
points  was  rejected  as  too  laborious  in  terms  of  manual 
handling  of  the  data  and  if  computed  would  require  a  modifi¬ 
cation  to  the  computer  program,  A  major  re-write  of  the 
program  would  also  be  required  if  the  analysis  was  to  consider 
the  unequal  replications.  This  dilemma  forced  a  re-evaluation 
of  the  objectives.  The  4x3x2x2  factorial  would  determine  in 
the  Investigation  of  differences  in  means,  differences  of 
approximately  5%  in  hit  probabilities.  This  type  of  dif¬ 
ference  Is  too  small  to  justify  the  development  of  stabilizers 
or  reflex  sights  for  small  arms  use.  Gross  differences,  on 
the  order  of  15”  or  20%,  would  indicate  that  Investment  in 
stabilizers  or  reflex  sight  development  could  have  a  meaning¬ 
ful  return  (in  terras  of  hit  probability).  Averaging  the 
replications  would  not  require  a  major  computer  program  re¬ 
write  and  this  method  of  "overcoming"  unequal  subclass 
numbers  was  finally  selected  with  the  full  awareness  of  the 
Iobs  in  precision. 

Figure  1  shows  the  moving  target  test  layout.  The  across 
target  firing  zone  was  100  meters  from  the  target  and  60 
meters  across.  Five  rounds  were  fixed  in  each  direction  by 
each  shooter  under  each  condition  of  the  across  target 
exercise. 

The  toward  target  course  was  120  motors  <n  length  and 
varied  in  range  from  170  meters  to  50  meters  from  the  target. 
Each  shooter  fired  10  rounds  in  the  120  meters  of  the  toward 
target  exorcise.  The  vehicle  speed  was  monitored  by  timing 
the  vehicle  between  the  "start  to  fire"  and  "-vase  fire" 
stakes.  Precise  vehicle  speed  was  difficult  to  maintain  and 
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controlled  rather  crudely  with  the  vehicle  leter  and 

a  stopwatch  over  each  120  meter  coursa. 

The  stationary  firing  was  accomplished  off  vehicle  in 
the  standing  position.  There  were  four  shooters,  each 
equipped  with  an  M16  Rirle,  reflex  sight  and  a  stabiliser 
and  the  necessary  ancillary  equipment. 

Figures  2,  3  &  4  show  the  data  accumulated  during  the 
tests.  These  are  averages  of  each  10  round  firing  group  of 
approximately  four  replications.  After  each  firing,  the 
target  was  coded  as  to  shooter,  toward  or  across  target, 
reflex  or  weapon  sight,  stabilized  or  unstabilized.  Figure 
3  is  a  date  sheet  consisting  of  data  taken  from  each  target. 

The  data  is  ln"cm"and  gives  the  coordinates  of  the  hit  with 
tha  origin  at  the  center  of  the  target.  This  particular 
data  sheet  shows  six  hits  in  a  ten  round  firing  by  shooter 
#4  using  a  reflex  sight  stabilized  when  moving  across  the 
target.  The  location  of  the  hits  in  this  case  represented 
data  which  was  not  used  during  moving  firing  analysis.  It 
was  planned  to  measure  mean  radius  for  each  ten  round  shot 
group.  This  was  impractical  since  the  location  of  the  missed 
rounds  was  unknown,  and  any  assumptions  about  the  rounds  that 
missed  would  most  certainly  bias  the  results.  Hit  probability 
was  considered  the  most  practical  measure  of  performance.  In 
the  stationary  firing,  all  the  rounds  hit  the  target  and 
presented  no  problems  in  determining  mean  radius. 

Figure  6  shows  the  factorial  analysis  printout  for  the 
moving  firing  across  target  data.  The  main  factors  are  S  - 
Sights,  U  -  Stabilization,  R  -  Speed,  and  G  -  Shooters.  You 
will  notice  that  the  F  ratio  for  sights  by  shooter  (SG)  inter¬ 
action  is  significant  as  well  as  the  main  factor,  speed.  The 
significance  level  is  .05  for  all  results.  When  an  inter¬ 
action  is  significant  no  inference  can  be  made  concerning 
the  main  factors.  The  significant  Interaction  indicates  that 
the  main  factors  are  not.  independent.  We  must  investigate 
the  main  effects  by  holding  either  sights  or  shooters  con¬ 
stant.  Sights  were  held  constant  for  this  analysis.  Figure 
6  also  shows  the  breakdown  of  the  across  target  analysis  into 
separate  eight  analyses.  The  reflex  sight  indicates  no 
significant  effects.  The  weapon  sight  shows  the  effects  of 
the  three  main  factors  significant.  A  covariance  analysis  of 
reflex  and  weapon  sighta  for  each  stabilization  mode  was 
performed  (Figure  7)  to  determine  if  a  reduction  in  residual 
mean  square  was  significant.  If  so,  covariance  analysis  would 
indicate  a  linear  regression  (1  df,  has  to  be  linear  if  signif¬ 
icant)  of  %  hits  versus  vehicle  speed.  The  reflex  sight,  based 
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upon  £h«  factorial  analysis,  indicated  no  significant 
differences  between  stabilization  modes,  therefore  the  co- 
variance  analysis  compared  both  stabilisation  modes  simul¬ 
taneously.  There  was  a  significant  reduction  in  residual 
due  to  regression  and  each  shooter's  slopes  wire  statistically 
parallel  with  the  Intercepts  squall  This  covariance  analysis 
concludes  that  a  single  regression  line  describes  the  per¬ 
formance  of  the  reflex  sight  stabilised  or  unatablllzed. 

The  equation  of  the  descriptive  line  Is  obtained  using  the 
combined  elope  and  the  Intercept.  The  examination  of  the 
weapon  eight  using  factorial  analysis  Indicated  a  significant 
difference  between  stabilisation  modes,  speed  and  shooters, 
therefore,  the  covariance  analysis  was  performed  examining 
each  stabilisation  level  separately. 

Figure  7  also  shows  the  analysis  for  the  wespon  sight 
stabilised  and  unstablllsed  across  target  firing.  In  the 
stabilized  mode  there  was  a  reduction  In  residual  variance 
due  to  regression;  however,  the  slopes  were  not  parallel  and 
the  Intercepts  were  not  equal.  This  Indicator  that  there  was 

a  significant  difference  in  performance  of  the  shooters  utilising 
the  weapon  sight  stabilised.  The  results  would  seem  difficult 
to  interpret,  but  it  can  be  readily  Been  that  In  moving 
across  the  target,  the  stabilizer  would  retard  the  angular 
motion  of  the  rifle  as  the  shooter  adjusts  for  the  movement 
of  the  vehicle  and  these  results  verify  the  difficulty  the 
shooters  had  In  firing  in  this  mode.  The  only  reason  this 
Is  not  apparent  with  the  reflex  sight  stabilized  is  that  the 
number  of  points  of  alignment  that  must  be  made  to  hit  the 
target  Is  S  for  the  reflex  sight  and  4  for  the  weapon 
eight  and  therefore  it  was  probably  easier  to  acquire  the 
target  with  the  reflex  sight  despite  the  retardation  of  the 
angular  motion. 

For  the  weapon  sight  unstablllsed  there  is  a  reduction 
In  residual  variance  using  covariance  (l.e.  there  is  linear 
regression).  The  test  for  paralllsm  shows  no  significant 
difference  indicating  the  slopes  are  parallel  and  the  inter¬ 
cept  analysis  shows  Intercepts  equal. 

This  analysis  concludes  that  a  single  regression  line 
describes  the  unstable  firing  results  of  the  shooters  when 
traveling  across  the  target.  Figures  8,  9,  and  10  illustrate 
the  results  of  the  across  target  firing. 
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Reflex  Sight:  One  linear  regression  describes  reflex 
performance  for  stable  and  unstable  and  all  shooters 
(Figure  8). 

Weapon  Sight: 

S tablet  Significant  difference  between  shooters 
covariance  analysis  -  linear  regressions  describe  perfor¬ 
mance  (Figure  9) . 

Unstable:  No  significant  difference  between  shooters 
covariance  analysis  -  one  linear  regression  line  describes 
weapon  sight  unstablo  performance  for  all  shooters  (Figure  , 
10) . 


A  comparison  of  reflex  sight  (stable  and  unstable)  per¬ 
formance  with  weapon  sight  stabilized  Indicated  no  signifi¬ 
cant  difference.  A  significant  difference  exists  between 
reflex  sight  and  weapon  sight  unstabilized.  This  Is  readily 
evident  by  comparing  the  levels  of  the  regression  lines. 
Figure  11  illustrates  these  comparisons  »  average  line  for 
each  condition. 


Figures  12  and  13  show  the  results  of  the  moving 
vehicle  toward  target  analysis. 

1  -  Factorial  Analysis  (Figure  12) :  Thera  are 
several  first  and  second  order  interactions  which  are 
significant!  Indicating  that  the  main  factors  are  not  Indep¬ 
endent  and  we  must  go  deeper  to  determine  the  causes  of  this 
significant  Interaction.  The  sights  were  examined  separately. 

2  -  Reflex  Sight:  Again,  a  first  order  interaction 
(sights  by  shooters)  is  significant. 

3  -  Weapon  sight  analysis:  Each  of  the  main  effects 
are  significant. 

4  -  A  covariance  analysis  of  reflex  sight  (Figure  13) 
shows  there  is  a  slight  reduction  due  to  regression  with  the 
slopes  parallel  and  intercepts  equal.  One  regression  line 
describes  the  performance  Independent  of  etabi lization. 


5  -  The  covariance  analysis  of  weapon  Sight  was  | 

examined  for  stabilised  and  unstabilized  conditions  separately 
and  the  results  arc  also  shown  by  Figure  13.  In  the  stable 
toward  target  condition  there  is  a  slight  reduction  using 
covariance  and  the  analysis  shows  slopes  parallel  and 
intercepts  equal, so  therefore  a  single  regression  line 
describes  the  results.  However,  In  the  unstable  toward 
condition,  the  slopes  are  parallel  but  the  Intercepts  are 
not  equal.  Therefore  there  is  a  significant  effect  caused 
by  the  shooters.  Figures  14,  15,  and  16  illustrate  these 
results: 

Figure  14  Reflex  Sight:  One  linear  presentation 
describes  the  results  (stable  or  unstable). 

i. 

Figure  13  Weapon  Sight  stabilised:  One  linear  t 

presentation  describes  the  results. 

Figure  16  Weapon  Sight  unstablllzed:  Shows  the 
significant  difference  between  shooters.  ; 

Figure  17  illustrates  the  comparison  betwesn  reflex 
and  weapon  sights  indicated  that  there  was  no  difference 
between  the  reflex  and  weapon  sights  stabilized,  and  both 
the  reflex  sight  and  weapon  sight  stabilized  are  significantly 
better  than  the  weapon  sight  unstablllzed  in  the  toward  tar* 
get  mode. 

ihe  factorial  analysis  and  covariance  analysis  for  Che 
stationary  firing  at  throe  ranges  wSb  performed  in  the  same 
manner  as  the  toward  and  across  target  analysis.  Figures 
18,  19,  and  20  summarize  the  results: 

-  There  was  a  significant  difference  between  pairs 
of  gunners  for  the  weapon  sight  stabilized  (Figure  18)  and 
unstablllzed  (Figure  19),  and  no  significant  difference  In 
performance  was  noted  due  to  stability.  (Compare  levels  of 
Figure  18  and  Figure  19) . 

-  During  stationary  firing  the  reflex  sight  again 
exhibited  no  significant  difference  in  performance  due  to 
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stability  and  thera  was  no  difference  in  the  performance 
of  the  shooters.  Again,  one  single  regression 'line  describes 
the  performance.. 

-  Figure  20  shows  that  the  performance  of  the  reflex 
sight,  stable  and  unstable,  is  significantly  better  (mean 
radius  lower)  than  the  weapon  sight,  stable  and  unstable, 
averaged  performance. 


CONCLUSIONS 

This  completed  the  analysis  of  the  test  data  end 
several  interesting  conclusions  ware  drawn: 

1  -  The  weapon  sight  performed  significantly  hotter 
stabilized  than  unstablllsed  on  moving  vehicles. 

2  -  There  was  no  difference  in  performance  using  a 
stabilizer  with  weapon  sights  compared  to  weapon  sights 
unstablllsed  during  stationary  firing. 

3  -  Stability  did  not  effect  the  performance  of  the  reflex 
eight. 

4  -  The  reflex  sight  (stsble  or  unstable)  performed 
significantly  better  than  the  weapon  sight  unstablllsed, 
and  there  was  no  difference  detected  between  the  reflex 
sight  (stable  or  unstable)  end  the  weapon  night  stabilized 
during  vehicle  firing. 

3  *  There  was  a  significant  improvement  in  mean  radius 
using  the  reflex  sight  (stabilized  or  unstablllsed)  versus 
the  weapon  aight  stabilised  or  unstablllsed. 

6  •  The  reflex  sight  has  a  tendency  to  normalise  the 
shooters  regardless  of  their  ability.  This  ’normalisation 
appear*  to  bring  the  level  of  ability  toward  tha  batter 
shooter.  This  normalization  aspect  is,  of  course,  Incon- 
elusive  since  the  levels  nf  the  shooters  wsre  unknown;  how¬ 
ever,  further  testing  of  uc  reflex  sight  to  verify  this 
aspect  was  recommended.  The  effects  of  a  atationary  shooter 
and  a  moving  target  with  the  stabiliser  waa  not  investigated; 
however,  based  on  the  results  of  across  target  weapon  sight 
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stabilised,  it  could  b*  postulated  that  stabilisation  would 
affect  the  hit  probability  Cor  this  type  of  fire. 

Saved  upon  the  results  of  this  test  and  in  conjunction 
with  the  UBAfAA,  Franfcford  Arsenal  is  developing  a  reflex 
sight  to  perhaps  replace  the  iron  sights  of  the  primary 
infantry  weapon. 
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CHARACTERIZATION  OF  BALLISTIC  EFFECTIVENESS 
BY  MAXIMAL  TRAJECTORY 


J.  T.  Wong  and  T.  H.  M.  Hung 
Systems  Res* arch  Division 
Research,  hevelopmant  and  Enginaaring  Diractorata 
HQ,  US  Army  Waapona  Command 
Rock  Island,  Illinois 


ABSTRACT 

Ths  purpose  of  chis  study  is  to  introduce  a  measure  by  which 
the  information  content,  poaaessed  by  the  trajectory  generated  by  a 
projectile,  pertinent  to  the  ballistic  effectiveness  of  a  weapon  aystam 
can  be  characterised. 

Having  this  concept  and  some  appropriate  mathematical  tools 
at  our  disposal,  a  few  relevant  results  are  obtained.  In  particular,  a 
characterisation  of  the  ballistic  effectiveness  of  a  weapon  system  in 
terms  of  tha  trajectory  information  is  given. 

The  results  obtained  give  aoma  indications  as  to  what  conditions 
the  data  enaemble  contains  all  the  Information  offered  by  trajectories, 
concerning  the  weapon  system  under  consideration.  Also,  a  formula  for 
the  total  of  maximum  recoverable  amount  of  information  possessed  by  a  set 
of  n  trajectories  (n  shots)  is  given  explicitly. 


The  remainder  of  this  article  has  been  reproduced  photographically  from 
the  author's  manuscript. 
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A  conventional  me choa  to  ootatr.  a  o.~~  o»  Ltrr.»r., —  *  l"  u~* 

data  experimentally  for  the  pcrposa  ci  analynl ng  the  ballistic  c-foc 
of  a  weapon  system  is  to  fire  the  weapon  upon  a  sufficiently  l-.vjc 
plane.  As  the  result,  Innas c  points  are  aecurea  ar.c  upon  wm an  a  w; 
established  analytical  procedure  is  th<p  applied  to  gain  some  cor.crc 
realisation  of  the  effectiveness  of  the  weapon  system  under  avaluac: 

The  purpose  of  this  study  is  to  establish  a  reasonao-e  maasura 
information  content  in  a  given  trajectory.  Hanes,  such  a  measure  t. 
applied  to  the  data  ensemble.  Subsequently ,  cn  the  basis  et  tan  t: 
and  soma  other  necessary  mathematical  notions,  a  rev  simple  ra-.u-i- 


obtaineu. 


Ihe  results  so  o'o  maimed  give  soma  ir.dlcat.oas  as  to  v.tai  ccnu-t.; 
the  data  ensemble  contains  all  the  information  cf ferae  by  a  tra^tttor; 
particular,  are  the  data  collected  from  the  "plane"  giving  si_  the  re. 


information  pertinent  to  the  capability  of  the  weapon  system. 


Best  Available  Copy 


and  U  is 


d (A, 3)  -  inf  {p(a,b)|aeA,  bcB) 

“  Inf  p(a,b) 
acA 
bcB 


where  "inf"  of  C*  is  the  greatest  lower  bound  cz  a  ncr.-er.pvy  tat  C  a; 
p  denotes  the  usual  Euclidean  distance  function  on  R* ;  i.e, , 

p:  R3  x  R3  ->■  [0,“) 
defined  by 

P(x,yl  -  J l  (x^y^2  ,  x.ycR5 

where,  for  example,  x^  is  the  i~  component  of  x. 

If  one  of  the  setn  A,B  is  the  null  set  p,  then 

d(A,B)  ■  “  , 

•  -  ■  *  *-*.-•  t  •  ^ oe  ».  o  n— c  i>ip  ty  a  o  s  o  ..  s  un  a  c i  a  ..  t  *aun 

there  exists  «a & e A  and  b.s3  sv.t.h  chat 

d  (A,SJ  “  pCoqjOq) 


Theorem  2.?.;  let  f  and  g  be  two  continuous  mappings  on  a  r.t 
space  a  into  a  metric  spate  a1 ,  Trie  set  A  of  all  points  xsE 
that  £(x)  »  g(x)  is  closed  in  E. 


Theorem  2.3;  If  A, 3  and  C  are  subsets  of  R3  such  that  ScrC, 


then 

d(A,B)  >_  d(A,C). 
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Definition:  For  every  left  .  lee  i.  he  a  continuon::  ;...v, nr."  cr. 

-  '  r.  ’  '  i  " 

R3  into  K  with  respect  to  the  usual  distance  rune  cions,  "hen  a  non- 
empty  subset  of  points  *  ■  (s^Cj.a^)  of  KJ  is  tailed  a  surface  S  i 

«  s  ■  £[  si 

ii)  S1  -  (scR3|C'i(s)  -  ki  ,  k^eS}. 

The  definition  for  a  surface  defined  here  is  different  iron  that  which  or. 
may  find  in  mathematical  literature,  for  which  n  ■  1  and  is  caf¬ 
fe  rent  i  able,  14,  p.  209),  for  example. 

Row  we  have  the  necessary  mathematical  tools  to  introduce  acme  r.av 
notions  and  proceed  as  follows. 

In  the  course  of  evaluating  experimentally  the  terminal  ball-otic 
effectiveness  or  capability  of  a  weapon  system  (particularly  .■  i.t.sj 
in  delivering  a  projectile  to  a  preassigned  fixed  "point"  (aa:.;in;'  pome), 
we  normally  collect  the  terminal  dispersion  data  by  piecing  an  aiming 
point  on  a  sufficiently  large  plana  upon  which  the  weapon  system  is  being 
fired,  Fallin  [3).  In  this  case,  the  dispersion  data  are  the  sat  of  all 
impact  points  on  the  plane,  with  respect  to  a  fixed  two- dir.er.sicr.il  car¬ 
tesian  coordinate  system. 

Every  impact  point  on  the  plane  possesses  three  intrinsic  properties 
of  the  corresponding  trajectory.  Obviously,  the  three  properties  are 
magnitude,  direction,  and  the  point,  which  is  itself  a  point  or.  the  tra¬ 
jectory.  The  vector  properties  of  an  impact  point  provide  r.oi  only  a 
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measure  or  closeness  of  a  given  trajectory  to  she  naming  pome  cur  also 
the  tree  .or  Chit  weapon  cystor.i. 

As  for  the  piano,  Its  function  in  to  pick  a  point  on  the.  tmjucco ry, 
ior  without  such  a  plana  we  do  not  know  in  general  the  vector  character 
ci  a  point  on  a  trajectory.  In  view  of  this  observation,  we  could  use  a 
surface  to  assess  the  function  of  a  plane  as  wall.  Since  every  olar.t  is 
a  burface  according  to  the  definition  above,  it  would  be  interesting  to 
find  out  what  kino  of  surfaces  would  ba  beet  to  use  instead  of  a  -.lane, 
r.'.-o  word  "best '  calls  for  soma  criteria  with  which  a  comparison  cert  be 
made  among  all  the  admissible  surfaces.  The  necessity  of  an  ap vronraace 
quality  measure  provides  part  of  the  motivations  for  defining  the  follower, 
concepts. 

The  foregoing  discussions  imply  that,  for  each  point  cr,  e  t.-a;attory 
there  corresponds  some  measurable  quantities  indicating  the  o.vr.br.ltf; 
a  weapon  system  in  delivering  a  projectile  to  the  aiming  uoinc.  A  -,si..t, 
lor  example,  on  a  trajectory  possesses  some  information  concernin';  alOiv- 
ness  to  the  aiming  point  --  consequently,  the  accuracy  of  the  system. 
Obviously,  tha  correspondence  between  each  point  on  a  trajectory  and  the 
corresponding  information  possessed  by  that  point  gives  in  a  natural  way  a 
function  between  two  sets.  Such  a  function  may  be  defined  as  follows! 

Definition :  Let  F  be  a  trajectory.  A  real-valued  function  :  or. 

F  into  the  reals  is  defined  by 


I(f)  "  1  +  p<£,9)  *  fcF 
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wh*.'*  P  is  usual  distance  function  0:1  the  euclidean  c.^iic-ipac , 
Ind  j  is  the  aiming  point  and,  for  convenience,  is  taken  to  be  the 
origin  of  the  coordinate  system, 

Ua  observe  that  the  function  1  is  well-defined  since  p  is. 

Also,  the  continuity  of  I  is  implied  by  that  of  p(  ,6)  on  a  subset  of 
.V.  Furthermore,  since  p  is  a  non-negative  reel-valued  function,  it 
follows  that  0  <  1(f)  £  1  for  any  point  f  belongs  to  a  trajectory  F 
Physically,  the  quantity  1(f)  is  a  measure  of  closeness  of  a  pair, 
f  on  the  trajectory  F  as  related  to  the  aiming  point.  Consequently, 
contains  knowledge  concerning  the  accuracy  of  the  weapon  system  under  co 
•  operation.  For  these  reasons,  we  may  interpret  the  quantity  Kt.)  f.a 
the  amount  of  information  associated  with  F  at  the  point  £.  ’.'he 
attainable  amount  of  information  .  •-•cessed  by  a  trajectory  F  is  giver. 

!“?  I(f)  1  +  inf  p (f , 3) 

“F  fcF 


Since  F  is  closed  and  bounded  —  the  direct  ima6.  oz  a  continuous  mspp 
on  a  compact  set  is  compact,  we  have 


Sup  1(f) 
fcF 


max  1(f)  k 
fcF 


1 

1  +  nin  (f ,6) 
fcF 


Therefore,  the  maximum  amount  of  information  possessed  by  F  is  recover 

.  *  „  *  .  ,,  Max  1(f)  -  I(f,) 

at  some  point  ffieF.  Symbolically,  £cy  0 

(The  existence  of  such  a  point  fQ  is  asserted  by  Theorem  2.1.)  As  for 
the  uniqueness,  ve  make  the  following  stipulation:, 
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,\  l  Tr.c i*u  c;-«isCa  ill  ..iOwt  or.c-  po^rit  x..c*  cuch  c!«r»L  If;:.  j 

• - — **  *  J  yJ 

!'>  the  :•••.:< i."’-"  recoverable  amount  of  information  of  F. 

7:.e  assumption  so  stated  is  not  indispensable ,  However ,  as  a  result 
<(  this  stipulation,  subsequent  analysis  is  oin.pl if icd.  In  fact,  the  ao- 
s.,Mp:ior,  is  reasonable,  for  in  the  noighbornood  of  the  aiming  point,  the 
raii-s  p-f  curvature  of  a  physically  realizable  trajectory  V  at  f  is 
Creator  than  p(fQ,6),  if  f  ffl;  that  is,  F  is  tangent  to  the  sphere 

■xt^lp^x.S)  -  p2(fo,0)} 


at  : . i : 


It.  view  of  tha  foregoing  remarks,  for  each  trajectory  F  there  exists 
anti  only  ona  point  fQ  belonging  to  F  ouch  that  X  assumes  its 
:..,xif.ur.  value  at  fQ. 

before  defining  the  last  concept  for  this  analysis,  vra  recall  for  e 
trajectory  F  with'  geF,  1(g)  ig  the  amount  of  infer  metier,  assigned  to 
;•  at  g.  Equivalently,  1(g)  is  the  amount  of  information  gained  about  ? 
by  observing  (sampling)  g.  •  Having  this  interpretation  in  mind,  we  make 
the  following  definition. 

Definition:  Let  F  be  a  trajectory.  A  real  valued  function  T  or. 

F  into  the  unit  interval  is  defined  by  the  formula 


1(g)  -  max  1(f)  -  1(g)  ,  geF 
fcF 

r.cte  tact  0  <_  1(g)  <  !(£.)  and  X(g)  “0  if  and  or.iy  if  g  ■  » 
It  is  t*acr  tact  1(g)  is  tne  amount  or  inrormatlcn  lost  concerning  F 
ju«  t*j  tbservir.g  g. 


d02' 


-irk;  If,  in  tho  definition  of  a  trajectory,  t^  is  taken  to  be 
the  tir.i  at  which  tho  projectile  leaves  tho  Muzzle  point  of  a  weapon,  chon 
the  nuzzle  velocity  f(tQ)  is  a  well  defined  quantity.  Hence,  f  is 
differentiable  at  t5  also.  Furthermore ,  the  definition  does  nor.  account 
lor  the  case  in  which  a  round  is  misfired  due  to,  for  instance.  Malfunction 
of  ;;.o  weapon.  In  such  an  event,  the  trajectory  is  the  null  sat  in  S'. 

;y  H"EVr.TS 

\fi:h  the  foregoing  concepts  at  cur  disposal,  vo  rr.oy  deduce  a  fuw 
irlvlal  consequences.  The  proof  of  tho  first  theorem  is  Immediate  and 
Is  enticed. 

Vheorem  1.1;  Lot  {ivjicf^}  be  a  cat  of  trajectories  and,  for  oath 

U(*  .  '-j  r  ho  the  unique  point  of  ?.  for  which  man  1(f)  “  I(f. ,).  "hen 
r.  it  1  £cFjL  i- 

S  K£l0>  «  «» 

i-1 

it  a  ad  only  if  everyone  of  the  n  trejectories  hits  the  aiming  point  0. 

before  we  proceed  to  the  second  theorem,  we  need  tho  following  tact 
at  our  disposal. 

Lemma ;  Every  surface  is  a  closed  set. 

Proof;  By  definition  a  surface  S  is  a  non-empty  subset  of  R*  such  J 

j 

l"l  I 

S  ■  |l  S  ,  for  some  positive  integer  n,  and  for  tef;  1 

i-1  1  n 

S1  -  (seR^^Cs)  -  ki;  k£eR} . 

1 
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by  definition  0^  is  a  continuous  function  on  R"  into 
constant  function”  defined  by 

Ci (o)  •  kt  ,  seR3 

is  also  a  continuous  mappia,,  on  IV  into  R.  12 y  •’iheoren  2 
{ecRJ  1^(6)  «  C,.  (s)  • 


is  closed  and  is  precisely  S^,  for  C.  (.•:)  “  k.. .  The  assert 
from  the  face  that  a  finite  ur.^or.  of  close.!  set  is  closed, 

Theorem  1.2:  Lot  {i'.iir.f.  ;  be  the-,  set  of  traiectori* 

- . -  e 1  n  J 

to  the  n  rounds  fired  froti  a  weapon  sys ter.  at  or.  aiming-  ?o, 
a  surface.  Furthermore,  let  A^  be  defined  by 

Ai  «  {ccR3|gcSfl?1}. 

Then 

n  n 

F  max  1(g)  -  I  max  1(g) 

i»l  gcAi  i*l  gcl^ 

is  a  necessary  and  sufficient  condition  for  no  information  i 
by  the  experiment. 

Proof :  Sufficiency 

A  CF  implies  inf  p(r,«)  *  inf  p(3,$)  by  Theorem 
R0?i  ScAi 


grrpg? 


I 


a. 

i 


f 


i 


i 


:ui\..iuatly 


fup  *  ;■',)  '  Sup 


jft >  ior  Ouch  i*\ 


Slip  1(a)  -  Sup  I<a)  _  0. 

ZF\ 


„•,  ;hc  1 cr.rr.a  above,  S  is  closed  and  so  is  ?\,  It  ioUc-is  tru-;  A  -  SoF- 

*  1  .  k 

closed*  donee >  su?reir.un\  is  the  sar/iG  us  r.‘.dv*ir.'»u»7i*  v/o  rviVu 
-ax  U&)  -  tr.nx  I  (3)  1  0. 

tV.e  hypothesis  implies  that  cor  each  i< f.^ 

~.a:<  -  s.>x  t(»)  -  0. 

Sir.  os  A. 


A.  r*  v  tor  each  the.',  there  exists  g^.rA.  such  tr 


i(Sic)  3  “-as  Us) 

fi£A, 


’.Ole  rose 


l(d,,)  a  ".ax  1(g)  -  I  (a,,)  a  0, 


Jlv 


«C'«  i 


1-3 


j£>,  cho  e::;ounc  o?:  in forma cion  concerning  F^  due 

/id  *  v  ^  u nd  c no  s  urn 


'  itf 


1 


■ 


vanishes  asserts  that  no  ini or nation  is  being  lost  by  the  experiment. 

\'.v  suppose  for  some  itfi^,  v •  than 

nax  Kg)  «  Sup  1(g)  «  0 
geA1  gsA1 

The  firs:  equality  follows  iron  the  race  choc  the  null  set  $  is  closed 
and  the  convention  of  the  supreme®  over  the  null  set  yields  the  last  eq¬ 
uality.  So  for  this  i, 

max  1(g)  -  max  1(g)  “  max  1(g)  >  0 
gsA^  ,  gtF^ 

The  3trioc  inequality  follows  from  a  remark  in  section  e  chut  «  •!  . 

Thus  if  A.  -  £  for  some  ir.P.  ,  then  the  hypothesis  r.tos  not  hold.  V,c 
i  n 

may  chan  deduce  that  the  conclusion  is  true. 

It  remains  to  prove  the  necessary  condition,  h'o  information  is  lost 

end  the  fact  that  ?.  ?i  $,  isO  >  imply  A,  y  <?.  Then  there  exists  r  .  eA 
i  n  i  i, 

such  that 

max  1(g)  -  I(rlfl). 
scAi 

Also 

0  -  max  1(g)  -  I(K.)  “  nax  Kg)  "  max  1(g) 
gs^  gfPi  ge^ 

Summing  over  the  last  term  on  1  gives  tho  assertion,  and  the  proof  of  the 
theorem  is  complete. 
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As  the  in-T.-sdiaia  eoaseruc.-.cox  or  tr.li,  result,  v_  h.ivu 
Corollary  4.3:  Suppose  A,  for  each  icf;  contains  or.c.  u.*.o  o 
-'•v.i.  there  exists  at  iocsc  ar.a  it.*;  such  that,  for  '  r A 

,  *  *  '  >«  *  u  1 

;  >  0  if  end  only  if 


n  A 

I  r.ax  I (£)  >  1  I(S,). 

i-i  feF .  i-1 


Corollary  A,d:  Lat  S  and  S*  ba  two  surfaces  such  shot  TO  5 
opar  subset  of  F^OS*  for  at  least  or, a  i.  Thao,  for  a  set 
octories , 


n  n  n 

Z  max  1(f)  £  Z  ir.ax  .1(f)  <.  a  s„ii,V  •>  (s  J 
i-1  feF.hS  “  i-1  feF.flS*  "  i-1  feF. 

11  X 


T.-worer.  4.5  Lc~c  ?  be  a  trajectory  no*  5  pc 
intersection  of  F  and  S,  Suppose 

r.ax  1(f)  <  r.:ax  1(f) 
fsA  feF 


there  exists  a  sequence  {h .  “  C  F  such  that 

1  i-1 

i)  r.ox  I(£)  -  lim  I(h^) 
feF  i-*« 

it)  there  is  a  posi.ti.va  integer  h'  such  that,  for  all 

i  >.  X,  I  (h , )  >  max  1(f) 

1  feA 
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Proof  :  Far  F  «  X  CO&iO  XT  u!e  •  civ.c  uhura  i.»  p.;in :  i 


i*«iCrl  Ci.^C 


r.ax  I  (£)  -  I(r  ’). 
•  _  «*  0 
Lb  f 


<ir.ca  F  is  closed,  the.:;  there:  cKia: j  a  MwO,ui;.co  1  h . ;  c.’  F  such  rF/.o 

1  i*I 

iia  h  ■  c , .  Also  va  hava  soar,  in  siccion  3  c’aac  c’u  iur.c:*.or.  I  is 

,  *  w 

continuous!  it  la  this  property  for  which  she  following  first  equality 
holds. 

lia  Kh  )  -  I  (11a  h,)  -  -  r.c:-:  1(f). 

i**«  i-«»  *  fc? 

It  rar-.aina  to  ba  shown  the  last  assertion  of  the  theorem.  If  A  *• 
ii)  holds  trivially,  since  F  j1  •>  and  we  may  pick  X  -  1.  5 'oppose 

A  s-'  0.  Again,  Thaorora  2.1  ir.plies  there  is  e  point  £,eA  such  choc 

ir.ax  1(f)  “  f(g ,). 
feA 

how,  by  hypothesis  wa  obtain  p(£  ,3)  <  e(g  ,i).  Also  wo  peck  uu 

0  c 

c  >  0  so  that  ?(f  ,6)  +  e  <  p(g  ,g).  For  this  o  and  cho  fee..  ch..c 
3  0 

lin  ht  -  e0 

i-«. 

’■'i  conclude  that  there  cxi  ts  a  positive  integer  X  such  that  for  oil 
i  l  X 

0  (h.  ,t  )  <  e, 

1  0 

dsir.j  th'csa  facea  and  r.ho  wall -known  .properties  oi  the  ir-u-uric  p,  wc 


p(hi,9)  <  p(hi,f(J)  +  <  u  +  p(f0,9)  <  p(g;,6). 
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obeai 


Proof:  For  F  y1  i  Theorem  2.1  os8irc8-th.it  there  is  u  pom:  f^cF 
such  that 

max  1(f)  ■  1(0. 
feF  0 

9* 

Since  F  is  closed,  then  there  exists  a  sequence  fh, }  CF  such  thee 

1  i-1 

lie  h.  “  f  .  Also  we  have  scan  ir.  section  3  chat  tha  function  1  is 
1 

continuous:  it  is  this  property  for  which  the  following  first  equality 
holds. 

lia  I(h.)  -  1(1 ia  h  )  -  1(f)  -  max  1(f), 

!-*•  1  '  i-w.  1  feF 

It  remains  to  be  shown  the  last  assertion  of  the  theorem.  If  A  •  §, 
ii)  holds  trivially,  since  F  ft  ♦  end  we  may  pick  N  »  1.  Suppose 
A  i  Again,  Theorem  2.1  implies  there  is  a  point  gQeA  such  chat 

max  1(f)  -  t(g  ) . 
fcA  u 

Sow,  by  hypothesis  we  obtain  p (f Q , 0 >  <  p(gQ,0).  Also  wo  nay  pick  ar 
e  >  0  so  thar  p(fQ,S)  +  c  <  p( Sq,6).  For  this  p  and  the  fact  that 

lira  h,  -  f. 
i**  u 

We  conclude  that  there  exists  a  positive  integer  N  such  that  for  all 
i  *  N 

p0vV  <  e* 

Using  these  facts  and  the  well-known  properties  of  the  metric  p,  we 
obtain 


P(hi,0)  <  P(hi,f0)  +p(f0,6)  <  c  +  p(f0,$)  <p(ge,«). 


Obviously,  1  +  p(h1,fl),  1  +  p(g0,»)  >  0.  It  follows  that 


1  +  Pi\t6)  *  1  +  p(gg,«')‘ 


KffJ  * 

<W 


foe  All  i  i,N,  where  N  depends  cn  t. 

Remark:  We  recall  that  in  section  3  eh*  definition  of  a  surface 
•  requires  every  surface  being  non-empty.  The  restriction  does  not  impose 
tny  further  constraint  on  the  results  in  this  section;  that  is,  the 
theorems  and  their  consequences  remain  valid  tor  the  case  in  which  S 
is  empty  and  in  particular  the  lemma  remains  true  since  the  null  set 
is  closed.  A  trivial  example  of  e  null  surface  Is  given  by 

3  ■  <xcR3,|pa(ax,bx(j)  -  -1,  a,  bcR,  xgcR*>. 

V.  COMCLtfSiOKS 

We  conclude  the  study  by  stating  explicitly  soma  of  the  observations 
deduced  from  the  forego*  v&  results. 

For  a  given  experiment, 
n 

£  D3X  1(f) 

i-l  feF1 

is  the  total  of  the  maximum  recoverable  amount  of  information  possessed 
by  e  set  of  n  trajectories.  Theorem  4,1  states  that  such  a  total  is 
equal  to  n  if  and  only  if  the  weapon  system  gives  a  perfect  performance 
concerning  its  capability  in  delivering  the  n  projectiles  qo  the  aiming- 
point. 


Thoorea  4.2  assarts  that,  in  tha  course  of  evaluating  tha  :a  rr.ir.cl 
* 

ballistic  effectiveness  of  a  weapon  based  on  the  result*  of1  ar.u.;.i,r,,  .... 
terminal  dispersion  data  collaccad  from  cho  surface,  or.  "ida.-.l  surra. *" 
to  ba  used  is  ona  which  parsarvaa  all  cha  in forrr.it tor.  ottered  by  the 
experiment;  equivalently,  an  icaal  aurfaca  it  cna  having  a  sat  or  r. 
trajectories  at  ita  tubaat.  A  simple  example  of  such  a  surface  S.  is 


ST  ■  LJ  {(x  ,x  ,x  )|Ri|x  -  A 5  x  ,  x  cR) 
1  left, 3}  123  1  23 


whtra 


•4* 

a  ».  min 

3  ■  rr.ax 

Aj  -  ,  J-0,1 

According  to  our  definition  of  a  surface,  S ^  is  not  e  surface,  tot  S{ 
is  a  union  of  uncouncably  many  planes.  Tharotoie,  a  perfect  surtacc  docs 
not;  exist.  Also,  cho  second  consequence  of  cha  theorems  tells  ua  chat 
there  is  a  positive  information  gained  by  extending  the  surface  being 
used  in  such  a  way  there  is  an  increase  in  the- data  ensemble  to:  sc  le^sc 
ona  trajectory. 

Finally,  the  last  result  concludes  chat  there  is  a  sequence  or  points 
in  “'a  given  trajectory  such  that  the  limit  of  the  image  ot  the  sequence 
under  1  is  precisely  the  maximum  information  possessed  by  the  trajectory. 
Also,  If  there  is  a  loss  of  information,  such  a  loss  can  be  reduced  by 
using  an  algorithm  to  recover  some  cf  the  loss. 
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Renark i  In  view  of  Sj  defined  above,  a  reasonable  lurtac*  i 

used  in  an  experiment,  is  one  which  consists  or  2r.rl  sufficiently 

planes  having  the  aiming  point  placed  on  the  center  or  one  p.i.'.t 
% 

regaining  2m  planes  erected  at  equidistant*  from  each  c:he:  cr.  s 
sides  of  the  center  plane. 
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ABSTRACT 


On  ths  basis  of  thraa-trlvariata  normal  distributions,  an 
sffactivaneas  modal  for  analysing  a  machine  gun  firing  bursts  on  volume 
targets  is  developed.  The  development  is  essentially  an  extension  of 
an  existing  model  beaed  on  three-bivariate  normal  distributions. 

A  regular  parallalplped  target  is  employed.  The  effectiveness 
criteria,  the  probability  of  incapacitation  and  the  expected  number  of 
hits  par  burst,  are  expressed  as  functions  of  hit  probabilities.  Finally, 
some  numerical  axamplaa  and  observations  are  given. 


The  remainder  of  this  paper  was  reproduced  photographically  from  the 
manuscript  submitted  by  the  author. 
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1.  INTRODUCTION 


Several  studies  have  been  conducted  for  analysing  affactlvanasa  of  a 
machine  gun  firing  bursts  upon  a  specified  target  [1,2,3,43 .  In  thasa 
studies  effectiveness  is  defined  in  terms  of  two  measurable  parameters: 

E(H)  and  P(I),  the  expected  number  of  hits  and  the  probability  of 
Incapacitation  par  burst,  respectively.  On  the  basis  of  these  effective¬ 
ness  criteria,  the  foregoing  investigations  have  rendered  some  basic 
models  for  effectiveness  analysis  by  using  a  two,  three  ot  four-bivariate 
normal  distributions.  The  models  so  obtained  have  provided  a  base 
for  analysing  effectiveness  on  area  targets. 

In  this  report  a  three-trivariate  noral  distribution  model  is 
developed  for  three  dimensional  target-volume  targets.  The  development 
is  essentially  an  extension  of  the  work  in  [3].  Thus,  no  further  physical 
justification  for  the  modeling  is  rendered  in  this  endeavor. 

As  was  known  in  [3]  the  dispersion  of  the  impact  points  from  a  burst  on 
area  target  revealed  some  interesting  measurable  phenomena.  In  particular, 
the  dispersion  data  ahowad  that  the  distribution  of  the  first  impact  point 
of  a  given  burst  differentiated  itself  significantly  from  that  rendered  by 
the  subsequent  projectiles.  For  this  reason,  it  was  assumed  that  the 
behavior  of  these  random  phenomena  can  ba  characterised  by  two  random 
vectors  X1  end  X2  respectively,  Furthermore,  the  offset  of  the  mean  of 
the  center  of  Impact  of  the  subsequent  projectile  from  that  of  the  first 
constituted  the  third  random  vector,  denoted  by  X3.  As  the  essential 
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premises  to  be  used  for  modeling,  it  is  assumed  choc  these  ob»*rv<bla  fact* 
can  bo  carried  over  .to  the  third  dimension  and  tach  random  vector  has  a 
mutually  independent,,  trivariate  normal  distribution. 

On  the  strength  of  the  foregoing  stipulations,  the  problem  is  solved 
for  cho  case  in  which  the  configuration  of  the  targe:  is  a  regular  paral- 
lolpipad.  In  the  formulations  of  the  effectiveness  parameters,  E(H)  and 
P(I),  are  attained  by  expressing  these  parameters  as  functions  of  P(P) 
and  P(S)  -  the  hit  probability  of  the  first  round  and  that  of  Che  sub¬ 
sequent  ones.  Finally,  numerical  examples  aa  well  as  some  interesting 
observations  are  given. 


2.  STATEMENT  of  THE  PR03UCM 
2.1  Modeling 

Referring  to  Fig.  1,  lot  tu  denote  a  fixed  poinc  in  E*> 

the  euclidean  3-spaco,  by  x^  ,  and  the  differential  of  x^  in  E3  to  bo  , 
dx|,d:<^,dXj  i  i.e.,  d:<^  “  dx^.dx^  dx^  .  A  solid  parallelepiped  target, 
as  shown  by  the  dotted  linos,  is  located  in  E  with  its  surfaces  inter¬ 
secting  the  coordinates  of  x^  *  (xj,Xj,x^)  at  the  points  (3^0,0), 

(s 2*0,0) , (0 , 3 j*0) , (0 i s^,0) , (0 ,0 , s j)  and  (0,0, s^). 
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3,  ygsmATigy 


Uc  us  bogia  by  noting  chat  tha  probability  dor.3ity  function 
can  bo  writ  can  in  a  produce  form,  43 

4.3  .  ....... 

P<*3>  -  ir  *(x}JW  j.oM 
i-1  xi  xi 

where 

J  "  1,2,3.. 

Tha  probability  that  the  first  round  hits  tha  target  T  ,  P(F) 
given  by 

PC')  -  f  p(x*)dx* 

JX 

where  tha  integral  symbol  denotes  tha  triple  integral  ovor  T  . 
Substituting  (3.1)  into  (3.2),va  obtain  P(F)  expressed  in  tarr. 
arror  functions  as  follows} 

-  jJartC.*}  .  .rf<aJl.l)]/, 

where 

°2i-l  ‘  “o'6!!-!  -  “x>>/»xl 

“2i  “Va-V)/,,.' 

*i 

for  i-  1,2,3  and  o  -1/^2 

0 


-117- 


.3  Of  vl 

(3.1) 

la  then 

(3.2) 

3  Of 

(3.4) 


Let  x1  designate  the  coordinates  of  tlic  first  round  of  a  burst, 
and  x  denote  that  of  the  subsequent  rounds.  The  coordinates  for  the 
offset  of  tho  nean  center  of  impact  of  the  subsequent  rounds,  which  is 


,  front  the  first  round  is  denoted  by  xJ 
,  1  ■  1,’2,3  or  simply  x3  **  yx2-  x3. 


3y  the  assumptions  stated  in  the  previous  section,  the  density 


functions  of  the  three  mutually  independent,  normally  distributed  random 

variables,  X.,*“  1,2,3  ,  can  be  written  as  $(x. ;u  ,a*  )  ,  where  u 

and  c*  are  the  mean  and  variance,  respectively',  of  the  distribution 
xi 

corresponding  to  the  random  variables.  It  follows  that  p  l  ,  1  ■  1,2,3, 

xi 

can  be  considered  as  the  actual  aim  point. 


2.2  The  Problem 

The  foregoing  considerations  enable  us  to  state  the  firs:  part  or 
the  problem  as  follows:  given  a  target  T  ,  defined  by  che  relation 


T  «  {xeE^Sj  <  Xj  i  ,  Sj  i  Xj  5.  84»  Ss  iyX3  5.  s6^  (2.2) 

ind  Che  input  parameters-  u  j  ,  i  •*  1,2,3  j  **  1,3  ;  ,  i,j  ■  1,2,3 

:ti  i:‘i 

i,  the  total  number  of  rounds  fur  the  burst.  and  i’Clhl)  ,  the  probability  o 
:apaeitacion  per  hit,  -  find  2(H)  end 
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3.  FORMULATION 


Lee  us  begin  by  r.oeing  that  the  probability  <!cr.3ity  functions  of 


can  be  written  In  a  produce  fora,  as 

1  3  1 
p(xJ)  -  it  «(xi;vt 

i-1  1  *  i  X1 


(3.1) 


where 


J  “  1.2,3.. 

The  probability  chat  Che  first  round  hies  tho  target  T  ,  P(F)  ,  is  then 
given  by 

PC-)  -  [  pCx^dx1  (3.2) 

'T 

where  the  integral  symbol  denotes  cha  triple  integral  over  T  . 
Substituting  (3.1)  into  (3. 2), we  obtain  P(F)  expressed  in  terns  of 
error  functions  as  follows: 

3  r  *r 


whora 


P(F)  -  ir^|arf(o2i)  “  crf^a2i-l0^2 
a2i-l  *  co(82i-l  " 


°2i  *  c0<82i  "  \\)/o  i. 

i  xt 

for  i  -  1,2,3  and  cfl  -  1//2 


117. 


For  a  given  y..2  tae  probability  that  a  subsequent  round  hits  T  , 


denoted  by  PCS j JJ  o)  i3 


p(s|jv1)  - 


p(x2)dx2 


(3.5) 


7 Via  probability  that  a  subsequent  round  hits  T  given  y.'  ,  denoted  by 
?(s|xJ)  ,  la 


PCS  Jx1)  -  |  pCxa>  { Vx2^«iux2 


0.6) 


where  dy  j  ■  dy  j.du  ■,  dy  j  and  the  integral  symbol  denotes  the  triple 

X  Xj  x^  x^ 

integral  over  E1  .  Using  (2.1)  and  (3.5)  in  (3.6),  we  can  rewrite 
(3.6)  tS  follow* i 


P(s|xl)  -  |  j.jP^x2”5*1)  p^2^dwx2  d*2 


(3.7) 


7he  probability  that  a  subsequent  round  hits  7  ,  denoted  by  P(3)  ,  in 

than 


P(3)  -  f  p  (x  • )  P(S|x5)dxl 
1  £3 


.Substitution  of  (3.7)  into  (3.3)  yields  the  following: 


P(S)  -  v  (erf(32l)  -  «rf <B  ^  ) >/2 
i“l 


(3.6) 


(3.9) 


wnere 


52i-l  "  b<S2i-l  “  \  m  v*l>\ 


fi2i  “  b<?2i  “  \l  “ 


2.  x  ,2^1/2 


for  i  -  1,2,3  and  a  -  (o;;j  +  o‘2  +  323) 

xi  ,{i  xi 
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The  expression  for  E(li)  can  now  bo  readily  obtained  for  a  buret 
of  X  rounds  by  substituting  aquations  (3.4)  and  (3.5)  in  thu  .o ’  /.-/in;] 

relations 

2(H)  «  Z  X.  ?(H. )  (3.10) 

iefl  •*  i 

where  PfH^)  is  tha  probability  choc  the  ich  round  in  a  burst  of  X 
rounds  hits  X  and  the  "characteristic  function"  X  on  the  index  sac 
(J  ■  (1,2,3,  •••  ,N}  is  dafined  by  tha  formula 


•  — ■  /  —  »  -  1  «  • 

r1 

if 

left 

x«'lo 

If 

an 

IC  follows  that 

E(H)  -  P(F)  +*(X-1)  P(S)  (2.11) 


To  seek  ?(t)  wo  proceed  in  the  following  manner.  It  is  wo  I known 
(5J  that  for  any  two  measurable  events 


p(xfT) +p(i.  I)  -p(i8) 


(3.12) 


where 

P Cl £  Ia)  denotes  tha  probability  that  the  first  projectile 
and  the  subsequent  projectiles  do  not  incapacitate 
the  target  T. 

P«fI8>  denotes  the  negation  of  P(I£Ia)  on  the  first  event. 

P(I  )  denotes  tha  probability  that  none  of  the  subsequent 

8  m 

projectiles  incapacitates  T. 


Hence, 

P(7)  -  p(is)  -  P(if  18) 
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For  a  given  »  Che  probability  that  none  o t  the  X-l  subsequent 

projectiles  incapacitates  T,  danotec  by  PfYJw^),  is  given  by 


-  J  U  -  ?<i!:-:t)?(:i1iux2)) 


(3.14) 


The  probability  P(IJ|x‘)  that  none  of  the  subsequent  projectiles 
incapacitates  T  given  x1  is  given  by 


Ptfjx1)  -  f  .  ?<x3)?(Ts!ux2>d!ix, 
^  E 


Ic  follows  that 


p<x  )  -  Y  (^J  t-p(i|ii)]k  n  [  j 

k-0  k  i-i  iE  ! 


iF« 


/r^2k 


::(erf(02i)  -*erf(32i_1)]kJ  dp^ 


where 


F«  "  +  \f'Cl 

\  “  0x{  +  °x3 


3 


21-1  "  (a2i-l  -  ",?>/*  Kl 


(3.15) 


(3.15) 


S 


21 

i  -  1.2,3. 


(s2i  “  Vo)/^2  CT  2 
i 


Here  we  have  used  the  assumption  that  for  a  given  hit  the  probability  of 

\ 

incapacitation  is  equally  probable;  i.c.  P(l|lQ  »  p(i|n  )  ,  for  -n  i£c, 


120- 


Vo  cvaulatc  P(I,  I  ),  wo  note  that 

Z  G 


whore 


p<if  Ts)  -  ?d!K)?(f  I.) 


PC?  T,)  -  PCx^PClJx1)^1 


.ftcr  soma  complications  on  tha  last  integral,  wo  arrive  ac 


N’-i  v  , 
/a-l\ 


PCI,  I.)  -  PCiiH)  z  (*YA)  (-?<i|h) )k  n 
k-o  ,  -  t-i 

-?F. 

f  k+ie ~ _ r~  -  erf(2  ))^ 


».[urf<a  S2l)  -  erf  (a  S,^))  |  duy2 


waera 


/a  /  /2*  o  i  o  > 
xi  xi  xi 


S2i-1  “  Ca2i-1 


cb  u-’  +  ®*t<v  “  v> 

xi  “i  Ai  xi  xi 


?v3  b  i  +  0vi(,,Jv2  - 

l  Xi  Xi  *i  1 


S2i  “  [3?.i 


1  “  1,2,3. 

P(I)  can  bo.  obtained  once  P(i)  in  (3.13)  la  docoi:.,i:-.od;  i,o. 


(3.1?) 

(3.18) 

(3.19) 


p(i)  “  l  -  P(T) 


(3.20) 


4.  s'jysaiCAT.  £.v.\.V:> r.ns 


!•? r  illustration,  a  regular  parallelepiped  target  o:  3  inches  is 

considered  first.  As  a  special  interest  wo  assume  that  the  target,  is 
teroed  in  at  the  center  of  its  front  surface  base  line  i.o,  -3  <  x,  <  3  , 
0  *2  i ^  i  and  0  <_  Xj  13  ■  A  rectangular  target  of  dimcnaloAs 

“J  i  *j  i  3  and  0  <,  Xj  <_  13  is  also  considered • as  a  special  case.  It 
provides  a  base  for  verification  with  that  given  in  (3], 


Table  1  gives  the  values  of  the  input  parameters-  S,  a  j  and  u  4 

xi  xj 


for  three  different  cases,  Table  2  shows  the  results  of  the  effectiveness 
parameters  for  the  two  targets.  The  value  of  is  taken  as  0.85 

for  finding  cha  values  of  P(I). 

Numerical  integrations  wore  computed  by  using  Simpson  Vs  rule. 

■» 

i'or  each  integration  .  equally  spaced  increments  ore  used  over  6; 


xi 


-lb 


ranges.  Any  numerator  value  les3  than  10  ,  as  well  as  any  value  for  the 

difference  of  two  error  functions  less  chan  10  is  taken  to  be  tore. 


■  5.  prsa'sssroys  &  coxctA^jcxi? 

An  effectiveness  model  for  machine  gun  bursts  on  volume  targets  has 
been  developed  with  throc-trivariate  normal  distributions  in  this  report. 
It  is  noted  that  for  targets  having  special  configurations,  the  cor¬ 
responding  models  can  be  readily  obtained  by  an  appropriate  coordinate 
transformational  An  area  target  model  is  obtainable  as  a  special  case. 
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For  instance,  Che  nodal  eo  obtained  for  regular  parallelepiped  target*: ie 
an  obvious  extension  of  the  threa-bivarlace  nodal  rendered  by  [3].  this 
fact  also  lead  us  to  note  that  P(F)  and  P(S)  for  e  voluno  target 
can  at  most  assume  only  one-half  of  the  corresponding  values  for  an  area 
target, 

the  conventional  notion  of  an  impact  point  becomes  rather  nebulous  whan 
carried  over  to  a  volume  target.  Further  pursuance  for  a  rigorousness 
development  of  thle  notion  may  bo  necessary.  The  raodor  le  referred  co  [<] 
for  such  an  affort. 

Since  the  essential  premises  used  for  this  modeling  in  extending  the 
random  phenomena  from  two  dimensions  into  the  third  dimension  Is  a  at rang 
one.  Voora  of  the  model  should  compriso  with  the  stipulations. 
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SIMULATION  OP  SUBSURFACE  NUCLEAR  EXPLOSIONS 
WITH  CHEMICAL  EXPLOSIVES 


Donald  S.  Burton,  and  Edwird  J.  Leahy 
Lawrence  Llvaraora  Laboratory,  Llwrarxe,  California 


ABSTRACT 


A  aarlaa  of  fiald  experiments  ara  balng  designed  and  thaoraeleal 
atudlaa  ara  balng  conduetad  to  aalact  a  chamlcal  explosive  and  to  davalop 
axploalva  charga  conflguratlona  and  aynthatlc  fallout  aatarlal  to  slau- 
lata  aubaurfaea,  aub-klloton  nuclaar  cratering  explosions,  The  studies 
and  experiments  ara  dlreetad  toward  data raining  tha  effects,  tha  typa 
and  dagraa  of  steaming  (full  stemming,  water  ct earning,  and  no  stem¬ 
ming)  have  on  the  alta  of  nuclaar  era tar a,  tha  vented  radioactivity,  ap£- 
tha  extant  of  the  raaultlng  fallout  pattern.  Such  Inforaatlon  for  witona- 
tlona  In  a  variety  of  geologic  media  la  required  If  nuclear  exploaivea  ara 
to  be  developed  aa  a  civil  and  ailltary  conatruotion  tool. 

The  problena  aaaoclatad  with  simulating  thaaa  affacta  through  tha 
uaa  of  chemical  exploaivea  ara  diacuosad,  and  partial  aolutlona  ara  pre- 
aantad.  Tha  theoretical  and  experimental  prograu  will  study  tha  relative 
affacta  of  tha  different  ataaaaing  conflguratlona  in  the  chamlcal  axploalva 
caaa  and  relate  tha  rasulta  to  the  nuclaar  caaa.  It  la  auggaatad  that 
thaaa  investigations  will  laad  to  optimum  design  criteria  fer  simulation 
axparlaanta  and  to  a  means  of  inferring  the  Information  which  cannot  be 
simulated. 


it 
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1.  INTRODUCTION 

The  development  of  nuclear  explosives  ae  a  tool  for  civil  and  military 
construction  requires  testing  In  a  variety  of  geologic  media  and  near¬ 
surf  rtce  emplacement  configurations.  Chsmlcal  explosives  are  e  conveni¬ 
ent  and  economical  way  to  model  nuclear  explosive  effects.  The  U.  S. 

Army  Engineer  Explosive  Excavation  Research  Office  (EERO),  Livermore, 
California,  a  part  of  the  U.  S.  Army's  Waterways  Experiment  Station  at 
Vicksburg,  Mississippi,  and  ths  Lawrence  Livermore  Laboratory  (LJ), 
Livermore,  California,  are  designing  a  series  of  field  experiments  to  be 
used  in  conjunction  with  numerical  modeling  calculations  to  develop  such 
a  simulation  technlqua.  Related  numerical  calculations  are  also  being 
conducted  by  Systems,  Sciencs,  and  Software  (SJ),  La  Jolla,  California, 

The  combined  effort  is  called  Project  DIAMOND  ORE.* 

Iu  the  development  of  the  technology  to  permit  employment  of  nuclear 
explosives  as  a  tool  for  use  in  both  civil  and  military  construction,  the 
influence  of  burial  depth  and  a teaming  material  on  phenomena  associated 
with  nuclear  cratering  must  be  determined,  Ths  construction  of  engi¬ 
neering  structures  such  as  railroad  cute,  storage  areas,  and  obstacle 
craters  requires  a  detailed  knowledge  of  ground  motion,  limits  of  material 
failure,  and  final  crater  alas.  Safety  considerations  require  knowledge  of 
airblaat  overpressures,  seismic  motions,  missile  throwout  ranges,  and 
radioactive  fallout  distribution.  The  DIAMOND  ORE  program  is  therefore 
directed  toward  modeling  as  many  of  these  effects  as  possible  with  the 
primary  emphasis  placed  on  crater  else  and  radioactivity  modeling.* 


*  Project  DIAMOND  ORE  IS  JOINTLY  FUNDED  BY  THE  Defense  Nuclear  Agency 
and  the  Office  of  the  Chief  of  Engineers,  US  Army, 

*  The  remainder  of  this  article  has  been  reproduced  photographically  from 
the  author's  manuscript. 


II.  SIMILITUDE  AND  CRATERING  PHENOMENOLOGY 

Thu  koy  to  the  successful  simulation  of  nuclear  cratering  eventa 
through  the  uao  of  high  explosive*  (HE)  la  the  understanding  of  the  crater* 
ing  phenomenology  which  results  from  both  energy  sources.  Although  tin 
quantitative  aspects  of  simulation  will  not  be  completely  known  until  the 
termination  of  Project  DIAMOND  ORE,  the  following  qualitative  aspects 
supply  a  starting  point  for  formulating  simulation  criteria. 

(1)  Early- time  Energy  Source  Behavior 

All  of  the  energy  from  a  nuclear  device  is  released  in  less  than  a  micro* 
second,  vaporizing  the  device  materials  and  its  canister.  (See  Point  1  in 
Figure  1).  As  this  high-pressure,  high- temperature  gas  pushes  on  the  walls 
of  the  emplacement  cavity,  a  strong  shock  wave  propagates  through  the  sur¬ 
rounding  rock,  depositing  sufficient  energy  to  cause  vaporisation.  Butkovlch^ 

i 

has  shown  that,  in  silicate  rock,  about  70  tone  of  rock  are  vaporized  per 

1 2 

kilo  to,',  of  yield  (1  kt  *  10  cal).  In  the  region  of  vaporisation,  the  re¬ 
sponse  of  the  rock  to  the  shock  is  such  thst  a  lower  limit  of  about  20%  of 
the  device  energy  la  lost  in  internal  heating  and  will  not  be  available  to 
do  mechanical  work  at  late  times.  As  the  shock  wave  propagates  outward, 
it  continues  to  melt  the  rock  and  shock-vaporize  water  contained  in  pores 
and  cracks.  Shock  vaporization  of  silicate  rock  and  water  occurs  at 
pressures  of  the  order  of  a  megabar  and  100  kilobars,  respectively. 

The  spherically  diverging  shock  wave  and  the  expanding  mass  of  vapor¬ 
ized  rock  and  water  are  the  principal  agents  in  the  formation  of  nuclear 
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crn  ter  a . 


In  the  cabo  of  chomical  explosives,  the  detonation  la  completed  in  a 
time  of  the  order  of  a  millisecond,  the  exact  value  depending  upon  the 
yield  and  type  of  explosive.  Because  the  average  pressures  within  the 
cavity  formed  by  the  gaseous  detonation  products  are  typically  less  than 
100  kilobars,  effectively  no  water  or  rock  is  shock  vaporised  In  the 
medium  surrounding  the  HE  cavity. 

The  HE  detonation  products  play  a  role  in  HE  cratering  which  is 
analogous  to  that  of  the  vaporized  rock  in  nuclear  cratering.  In  a  model* 
ing  experiment,  complete  similitude  cannot  be  achieved  if  the  pressure- 
volume  adiabat  for  the  chemical  explosive  detonation  products  does  not 
coincide  with  that  of  the  nuclear  cAvity  gas.  This  condition  obviously 
cannot  be  met  because  the  nuclear  gas  does  work  on  its  surroundings  in 
the  pressure  range  between  100  kb  and  1  Mb,  while  the  HE  does  not. 
Because  of  the  large  initial  pressure  difference,  the  coincidence  of  the 
HE  and  vaporized  rock  adiabats  in  the  high  pressure  regions  can  no  longer 
be  used  as  a  similitude  requirement. 

However,  successful  simulation  of  the  cratering  mechanism  hinges 
upon  transferring  not  the  same  total  energy  to  the  medium  as  the  above 
argument  would  require,  but  rather  the  same  kinetic  energy.  The  peak 
pressure  and  response  of  the  cratering  medium  to  the  shockwave  governs 
the  amount  of  energy  lost  in  shock  heating  and,  consequently,  the  amount 
of  kinetic  energy  transferred  to  the  medium.  The  response  of  a  given 
material  to  a  shock  is  characterized  by  a  release  path  or  Hugoniot  In 
pressure-volume  space  (see  Figure  2).  In  an  HE  detonation,  a  region  of 
material  near  the  HE  cavity  is  shocked  from  point  I  to  A,  receiving  energy 
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represented  by  the  area  LAB.  As  the  pressure  decreases,  the  material 
unloads  along  the  Hugoniot,  doing  work  ABF  on  the  surroundings  and  re¬ 
taining  energy  AFI  as  Internal  energy  or  "waste  heat".  In  a  nuclear  deto¬ 
nation,  the  medium  is  subjected  to  a  much  greater  pressure  and  receives 
greater  energy  JCD.  However,  most  of  this  energy  is  lost  as  waste  heat 
ICF.  Consequently,  although  the  nucle&r  case  involves  much  greater 
pressures  than  the  HE  case,  only  a  relatively  small  amount  of  additional 
energy  (BCD)  is  made  available  for  the  cratering  process. 

In  other  words,  the  very  high  pressure  shock  wave  resulting  from  a 
nuclear  detonation  does  not  couple  efficiently  with  the  medium,  giving  rise 
to  large  energy  losses  through  shock  heating.  (The  presence  of  walls 
allows  a  small  but  significant  quantity  of  this  heating  energy  to  eventually 
return  to  the  system  in  a  mechanically  usable  form).  As  the  shock  wave 
propagates  outward,  this  energy  loss  results  in  the  rapid  attenuation  of 
the  shock  peak  to  levels  attainable  with  high  explosives.  Because  of  the 
lower  loss  rate  of  the  HE  shock  wave,  cratering  similitude  can  then  be 
achieved  even  though  the  adiabats  for  the  vaporized  rock  and  the  HE  deto¬ 
nation  products  aro  not  identical  in  the  high-pressure  region.  Furthermore, 
because  of  the  difference  in  total  energy  loss  between  the  two  caseB,  the 
HE  charge  can  be  selected  to  have  a  much  lower  energy  yield  than  the 
nuclear  device.  However,  it  must  be  understood  that  effects  which  depend 
upon  tho  material  temperature  and  effects  occurring  in  the  region  very 
noar  the  cavity  cannot  be  modeled. 

(2)  Spallation 

In  both  the  nuclear  and  HE  cases,  as  the  shock  wave  intersects  mater¬ 
ial  layers  and  finally  the  surface,  it  is  reflected  back  towards  the 
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cavity  in  fhc  form  of  rarofactions  or  tensile  wivea,  Aa  the  tensile  wave 
passes  downward  through  the  medium)  the  materialibroken  by  compression, 
tension,  or  shear,  spalls  off  in  layers  which  then  execute  an  approximate 
ballistic  trajectory.  (See  point  2  in  Figure  1). 

The  depth  from  which  material  is  ejected  by  spall  la  a  function  of 
the  material  strength  and  kinetic  energy  transferred  to  the  mound  by  the 
shock  wave.  In  simulating  the  spall  phenomena,  primary  consideration 
must  be  given  to  matching  not  the  peak  particle  velocities  which  are 
short-lived  (except  at  the  surface),  but  rather  the  residual  particle  veloc¬ 
ities  (those  particle  velocities  which  remain  after  the  shock  peak  has 
subsided)  throughout  the  mound .  This  criterion  is  approximately 
equivalent  to  matching  the  total  mound  kinetic  energies  produced  by  the 
shock  front.  Preliminary  calculations  suggest  thst  only  15  to  20%  of  ths 
total  nuclear  device  yield  is  converted  to  mound  kinetic  energy  prior  to 
the  gas  acceleration  phase,  whe  reas  this  figure  may  be  as  high  as  30  to 
40%  for  an  HE  such  as  nitromethane. 

( 3 )  Ground-Shock-Induced  Airblast 

When  the  shock  hits  the  free  surface,  the  top  surface  layers  will  spall 
off  at  approximately  twice  the  peak  velocity  of  the  particles  at  the  same 
distance  from  shot  point  but  not  at  the  surface.  (Point  3  in  Figure  1). 

The  overall  motion  of  the  ground  surface  acts  as  a  piston  and  couples 
with  the  atmosphere,  producing  "ground-shock-induced"  airblast.  The 
simulation  of  this  effect,  then  requires  that  the  HE  and  nuclear  explosions 
produce  the  same  peak  particle  velocities  at  the  ground  surface.  This 
criterion  conceivably  conflicts  with  the  previously  stated  kinetic  energy 
criterion.  Although  it  may  be  possible  to  match  both,  such  circumstances 
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depend  upon  the  rapid  attenuation  of  the  sharp  peak  of  the  nuclear  ahock 
wave  relative  to  the  peak  of  the  HE  shock  wave.  Whether  sufficient  attenu¬ 
ation  has  occurred  by  the  time  the  wave  strikes  the  surface  depends  upon 
the  material  properties  and  the  shot  depth. 

(4)  Seljmtc  Effects 

As  the  shockwave  (and  associated  particle  motion)  propagates  outward 
in  die  horizontal  direction,  it  attenuates  rapidly  with  distance  because  of 
the  spherical  divergence  and  because  of  energy  deposition  in  the  medium . 
Dispersive  effects  round  off  the  sharp  peak  of  the  shock  front  through  the 
attenuation  of  high  frequencies;  and,  as  the  shock  amplitude  decreases, 
the  material  begins  to  behave  elastically.  The  disturbance  continues  to 
move  outward  as  a  seismic  pulse  traveling  at  acoustic  velocities.  Because 
of  the  dispersion,  the  amplitude  of  the  particle  motion  is  more  directly 
related  to  the  residual  particle  velocities  than  to  the  peak  particle  velocities. 
Because  of  the  dispersion,  the  peak  and  residual  particle  velocities  will  be¬ 
have  in  a  similar  manner  at  large  distances.  For  this  reason,  it  is  antici¬ 
pated  that  modeling  of  seismic  motion  will  be  achieved  by  a  similitude 
based  upon  the  kinetic  energy  criterion. 

(5)  Gas  Acceleration 

As  the  rarefaction  impinges  on  the  cavity,  the  stresses  in  the  rock  at 
the  roof  of  the  cavity  are  relieved;  and  the  cavity  begins  an  accelerated 
asymmetrical  growth  toward  the  surface,  producing  the  so-called  gas 
acceleration  phase  of  the  cratering  process.  (Point  5  in  Figure  1. )  The 
magnitude  of  this  effect  depends  primarily  upon  the  time  at  which  the  rare¬ 
faction  intersects  the  cavity  (and  therefore  the  shot  depth),  the  cavity  pres¬ 
sure  at  this  time,  and  the  expansion  of  the  cavity  gas  below  this  pressure 


(i.o.,  tho  low-pressure  gas  adlabat).  As  the  cavity  expands ,  the  pressure 
and  lomperaturo  decrease.  Any  drop  in  temperature  below  about  2000°  K 
will  be  accompanied  by  the  partial  condensation  of  the  vaporised  rock  gas, 
bringing  about  an  accelorated  drop  in  pressure.  Such  an  effect  will  nof 
be  experienced  by  the  essentially  noncondensable  HE  detonation  products. 
Preliminary  calculations  indicate  that  for  the  same  yield  the  pressure  and 
available  energy  in  a  nitromethane  cavity  gas  at  relatively  late  times  may 
be  several  times  that  of  the  nuclear  cavity  bases.  For  nuclear  detpnations 
in  wet  materials,  the  steam  produced  by  shock  vaporization  will  behave  as 
a  noncondensable  gas  and  will  give  rise  to  somewhat  higher  late-time  pres¬ 
sures  than  in  comparable  dry  materials. 

As  indicated  earlier,  the  simulation  of  spallation  requires  the  matching 
of  tho  mound  kinetic  energies  rather  than  the  high-pressure  gas  adlabats. 
The  simulation  of  the  gas  acceleration  phase,  however,  requires  the  co¬ 
incidence  of  the  adiabats  at  the  low  pressure  end.  While  these  are  not 
necessarily  mutually  exclusive  criteria,  they  may  not  be  simultaneously 
realizable  with  present-day  explosives.  The  modeling  of  either  spalla¬ 
tion  cratering  or  gas-acceleration  cratering  presents  no  conceptual  prob¬ 
lems.  However,  the  modeling  of  nuclear  events  in  which  both  processes 
are  significant  may  present  many  difficulties .  Certainly,  it  is  always 
possible  to  select  an  HE  yield  which  will  produce  a  crater  of  the  same 
volume  as  that  produced  by  a  spallation /gas -acceleration  nuclear  shot. 

But,  some  of  the  secondary  effects  will  not  be  properly  modeled;  the  ratio 
of  radius  to  depth,  for  example,  may  be  in  error.  The  magnitude  of 
these  errors  can  only  be  found  through  HE  experiments  and  extensive 
numerical  modeling. 
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( 1  •)  Uj'JJlLIi  hL  Kolnlivo  Crater  .Sir.tm 

In  very  shallow  shot*,  the  overburden  spells  off  quickly  so  that  tho 
cavity  gases  vent  and  do  work  on  the  atmosphere  rather  than  the  mound. 

In  somewhat  deeper  shots,  the  early  spallation  is  reinforced  by  the  latpr 
gas  acceleration  mechanism.  As  the  depth  of  burst  is  increased  etill 
further,  the  shock  wave  becomes  weaker  at  the  surface,  causing  less 
spallation;  and  gas  acceleration  begins  to  play  the  dominant  role.  Although 
it  is  commonly  believed  that  maximum  crater  siae  is  obtained  for  shot 
depths  at  which  spallation  and  gas  acceleration  contribute  equally,  this 
has  not  been  firmly  established,  and  only  experiment  or  precise  numerical 
modeling  can  answer  this  question  for  a  given  medium. 

Thus,  the  spallation  and  gas  acceleration  mechanisms  combine  to 
produce  a  variation  in  crater  dimensions  with  depth  for  a  particular 
energy  yield  and  material.  Generally,  the  crater  radius  and  depth  peak  at 
approximately  the  same  burial  depth  which  is  termed  the  "optimum  depth  of 
burst".  Because  of  the  different  emphasis  placed  on  spallation  and  gas 
acceleration  by  the  different  energy  sources,  the  "cratering  curves"  for 
a  high  explosive  and  a  nuclear  explosive  (or,  for  that.matter,  other  high 
explosives)  cannot  be  expected  to  have  the  same  shape,  and  therefore  the 
same  optimum  depth  of  burst.  As  has  been  indicated  several  times,  the 
best  approximation  should  occur  when  the  same  kinetic  energy  is  imparted 
to  the  mound  by  the  shock  wave  and  when  the  cavity  gases  have  the  same 
late-time  pressure.  If  this  criterion  is  met,  the  cratering  curves  should 
coincide,  providing  the  HE  and  nuclear  yields  are  redefined  to  take  into 
account  tho  energy  loss  due  to  shock  heating  of  the  medium.  This  redefini¬ 
tion  is,  of  course,  media-dependent  to  some  extent.  If  the  condition  cannot 
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bu  mot,  one  mutt  resort  to  simulating  a  fixod-yiuld  nuclear  uxploaivo  »l 
different  depths  with  HE  charges  of  varying  yields  which  will  depend  upon 
the  predominant  cratering  mechanism  at  that  depth.  Use  of  this  approach 
will  result  in  the  imperfect  simulation  of  some  of  the  secondary  nuclear 
effects. 

(7)  Missile  Haaarda 

As  the  maximum  missile  ranges  are  usually  produced  by  the  larger 
blocks  of  material  which  have  been  spalled  off  the  surface  by  the  shock 
wave,  simulation  involves  the  same  criteria  and  consequences  as  dis¬ 
cussed  for  ground* shock- induced  airblast.  An  exception  to  this  may  occur 
in  soils  for  which  the  top  layers  are  significantly  weaker  than  the  deeper 
materials.  Under  these  conditions,  the  more  energetic  missiles  may 
originate  frornjjae  deeper  layers. 

(8)  Gas -Vent-Induced  Airblast 

The  effects  of  the  cavity  gas  extend  beyond  providing  a  boost  to  the 
mound.  If  the  mound  breaks  up  before  the  cavity  pressure  has  dropped 
to  one  atmosphere,  the  cavity  gases  do  work  on  the  air  to  produce  gas- 
vent  airblast.  (Point  7  in  Figure  1),  Simulation  of  this  effect  involves 
reproducing  the  nuclear  cavity  pressure  and  energy  at  vent  time  and,  con¬ 
sequently,  matching  the  adiabats  in  the  low-pressure  range.  This  presents 
many  problems,  particularly  in  shsdlow  shots  in  which  spall  is  the  primary 
cratering  mechanism.  That  is  to  say,  matching  the  adiabats  to  reproduce 
gus-vent  airblast  may  give  rise  to  a  mismatch  in  the  spall-producing  por¬ 
tion  of  the  adiabats  possibly  to  the  extent  of  affecting  the  vent  time. 
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(9)  Venting^ 

In  nuclear  events,  the  behavior  of  the  cavity  gaa  ie  Intimately  related 
to  the  venting  of  radioactive  fieaion  products  and  induced  radionuclides 
which  later* appear  as  fallout.  As  the  mound  grows  and  begins  to  dissociate, 
those  radioactive  cavity  gases  which  are  still  uncondensed  filter  or  are  in¬ 
jected  through  fissures  in  the  mound.  The  violence  with  which  the  gas  is 

i 

released  to  the  atmosphere  will  depend  upon  the  cratering  mechanism  at 
the  time  of  release:  (a)  if  gas  acceleration  is  the  predominant  mechanism, 
then  a  large  cavity  pressure  at  late  times  can  be  expected-  to  give  rise  to 
a  relatively  large  vent;  (b)  in  a  shallow,  spall -produced  crater,  the  spall 
depth  can  extend  into  the  cavity  region.  Under  these  circumstances,  the 
venting  should  be  large  and  may  be  relatively  Independent  of  cavity  pres¬ 
sure.  On  the  other  hand,  if  spall  is  the  primary  cratering  agent  and  the 
spall  depth  does  not  extend  into  the  cavity  region,  then  a  relatively  small 
vent  is  to  be  expected.  Cavity  gas  will  still  be  forced  through  fissures  by 
the  late-time  cavity  pressure,  but  to  a  lesser  extent  than  in  the  previous 
cases.  Clearly,  the  complete  simulation  of  the  venting  mechanism  re¬ 
quires  comparable  late-time  cavity  pressures  and  therefore  similar 
adiabats  for  the  HE  and  vaporized  rock  in  the  lower  pressure  region, 
although  it  is  possible  that  this  condition  may  be  overly  restrictive  in  that 
the  venting  mechanisms  may  not  be  sensitive  to  the  exact  cavity  pressures 
involved. 

The  cavity  pressure  at  vent  time  can  also  have  a  significant  effect 
on  the  composition  of  the  fallout.  This  arises  because  the  radioactive 
fission  products  and  induced  activities  entrained  in  the  cavity  gas  behave 
in  chemically  and  physically  varied  manners.  The  "refractory" 


radionuclitli'H  belong  to  maos  chains  which,  on  tho  averago,  form  oxUlus  with 
relatively  high  c.umlunsallon  tumporaluree  between  about  1100  and  1700°C^V 

j *** 

If  thu  cavity  tcrmperaturo  drops  below  this  rang*  prior  to  venting  (cor- 
responding  to  a  lower  cavity  pressure  and  less  violent  vent),  the  refractory 
radionuclides  will  condense  along  with  the  vaporised  rock  to  form  volume* 
distributed  droplets  which  are  preferentially  surface-deposited  on  the 
large  particles  of  the  dissociating  mound.  Radionuclides  belonging  to  the 
"volatile"  mast  chains  will  not  condense  until  much  lower  temperatures 
are  reached..  Presumably,  th*  condensation  of  volatiles  will  occur  at  a 
late  time  when  tftere  is  a  preponderance  of  fine  particles,  so  that  the  vol¬ 
atiles  will  be  preferentially  deposited  on  smaller  particles  than  the  re¬ 
fractories.  This  difference  in  behavior  is  termed  "fractionation",  and 
gives  rise  to  a  distribution  of  radionuclides  in  the  cloud  and  fallout  which 
differs  markedly  from  that  distribution  produced  by  Aie  nuclear  source. 

If,  on  the  other  hand,  the  cavity  pressure  ie  high  at  the  time  of  vent, 
neither  refractories  nor  volatiles  will  be  afforded  the  opportunity  to  con¬ 
dense  prior  to  being  injected  through  the  mound  into  the  atmosphere. 

Under  these  conditions,  there  will  be  little  preferential  condensation  on 
particles  of  any  given  size,  and  no  marked  fractionation  will  occur. 

(Notable  exceptions  to  this  are  radionuclides  which  behave  predominantly 
as  noncondensable  gases  . ) 

Consequently,  it  ie  legitimate  to  speak  of  a  "vent  fraction"  (or  frac¬ 
tion  of  produced  radionuclides  which  are  vented  into  the  cloud  and  fallout) 
in  the  case  of  violent  venting.  However,  in  the  case  of  a  low-pressure 
vent,  one  must  speak  of  a  vent  fraction  for  each  individual  mass  chain 
or,  at  beat,  a  separate  vent  fraction  for  refractory,  volatile,  and 
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As  it  it  obviously  impractical  to  attempt  simulation  o!  a  nuclear 
fallout  pattern  with  high  explosives,  the  modeling  must  be  confined  to 
reproducing  a  parameter  such  as  the  vent  fraction(s)  which  will  allow  an 
equivalent  fallout  pattern  to  be  generated  by  numerical  techniques.  In 
the  case  of  high-ora  s  sure  venting,  this  simulation  can  be  accomplished 
in  a  relatively  straightforward  manner.  In  principle,  one  need  only  em¬ 
place  some  sort  of  traceable  material  uniformly  throughout  the  high  explo¬ 
sive  and  then  recover  it  from  the  region  outside  the  crater.  In  practice, 
there  are  many  problems  which  must  be  overcome;  these  will  be  des¬ 
cribed  in  more  detail  later.  Again,  there  is  the  everpresent  restriction 
that  the  HE  detonation  products  and  die  nuclear  cavity  gases  must  have 
approximately  the  same  late-time  pressure.  Simulation  of  low-pressure 
venting  necessarily  involves  the  use  of  different  types  of  tracer  materials. 
It  is  doubtful  that  particulate  tracers  will  satisfactorily  simulate  the 
volatiles  under  these  conditions.  Ideally,  one  needs  tracers  which  will 
go  through  the  same  physical  phases  at  the  same  times  as  the  radionuclides 
but  at  the  much  lower  HE  temperatures.  At  the  present  time,  consider¬ 
ations  of  this  nature  appear  to  be  beyond  the  state-of-the-art. 

(10)  Cloud  Formation 

Cratering  events  normally  exhibit  a  cloud  structure  comprising  of 
a  main  cloud  and  a  bate  surge.  This  structure  is  intimately  related  to 
the  distribution  of  fallout  produced  by  nuclear  events.  With  the  complete 
dissociation  of  the  mound,  the  material  is  ejected  along  ballistic  trajec¬ 
tories.  The  primary  mechanism  for  the  formation  of  the  base  surge  is 
thought  to  be  the  potential  energy  in  the  euepended  aerosol  formed  from 
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tho  loflod  mound  material.  The  formation  ia  interpreted  physically  as  a 

gravity  flow  of  the  eeroeol,  during  which  energy  is  transformed  from 

potential  energy  to  kinetic  energy  of  the  aerosol  and  then  to  work  done 

f*l 

in  displacing  environmental  alrl**J.  Correct  simulation  of  the  base  surge 
then  follows  from  the  correct  simulation  of  the  late- time  mound  kinetic 
energy  whether  caused  by  spallation  or  gae  acceleration. 

The  formation  of  the  main  cloud  is  keyed  to  the  cavity  pressure  at 
vent  time;  and.  the  absence  of  this  cloud  can  be  expected  If  a  violent  vent 
does  not  occur.  With  the  violent  release  of  the  cavity  gases,  the  main 
cloud  rises  from  ground  aero  and  grows  primarily  in  the  vertical  dlrec- 
don,  entraining  fine  particulates  from  tha  mound.  Tho  growth  is  Influ¬ 
enced  by  the  initial  momentum  of  the  hot  cavity  gas  bubble,  buoyant  forces 
on  the  bubble,  the  internal  friction  produced  by  turbulence,  the  external 
friction  and  pressure  exerted  on  the  bubble  by  the  environment,  and  the 
entrainment  of  mass  in  the  form  of  air  and  particulates. 

If  the  main  cloud  were  formed  by  an  adiabatic  process,  then  the 
matching  of  the  cavity  pressures  and  adiabata  would  bo  tha  key  factor  in 
arriving  at  an  HE  simulation.  However,  in  reaching  stabilisation,  the 
cloud  exchanges  energy  with  the  environmental  air.  As  the  HE  cannot 
attain  the  temperatures  of  the  nuclear  cavity  gaa,  it  is  doubtful  that  there 
exists  a  main-cloud  simulation  criterion  which  is  compatible  with  the  pre¬ 
viously  stated  cratering  simulation  criteria. 

(11)  Effects  of  Stemming  Materials 

(a)  Solid  Stemming.  Insofar  as  the  material  with  which  the  emplace- 

i 

ment  hole  is  stemmed  is  a  reasonable  approximation  to  the  in- situ  material, 
the  stem  will  not  have  a  profound  effect  upon  the  early  spallation  stages  of 
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crater  dovolopmsnt.  Howovsr,  aa  the  cavity  expands,  tha  original  borahola 
also  enlarges,  first  near  the  charge  and  later  near  the  ground  surface.  Solid 
stemming  may  tend  to  lose  contact  with  the  wall  surfaces,  thereby  per* 
mitting  premature  venting  of  the  cavity  gases  with  subsequent  loss  of 
energy  available  for  the  gas-acceleration  stage.  This  effect  can  of  course 
be  reduced  by  special  emplacement  designs.  If  other  simulation  criteria 
are  met,  it  is  likely  that  solid-stemming  materials  will  behave  comparably 
in  the  HE  and  nuclear  cases. 

(b)  Liquid  Stemming.  The  use  of  water  stemming  ha#  been  tested  using 
high  explosives  with  the  possible  result  of  increased  cratering  efficiency. 

This  has  been  attributed  to  the  ability  of  liquid  stemming  to  flow  laterally, 
maintaining  contact  with  the  walls  of  the  borehol^.  This  concept  cannot 
be  simply  extended  to  the  nuclear  case  because  different  physlcsd  principles 
come  into  play,  and  one  can  only  speculate  on  the  consequences.  As  indicated 
earlier,  the  amplitude  of  toe  nuclear  shock  wave  is  sufficient  to  shock- 
v&porize  water  considerably  beyond  the  limit  of  rock  vaporisation.  This 
will  cause  the  formation  of  a  steam  bubble  in  the  lower  portion  of  the  stem 
which  might  contribute  to  toe  premature  ejection  of  toe  comparatively  low- 
density  water  stem  with  the  concommitant  early  venting  of  the  radioactive 
cavity  gases.  Although  the  design  of  en  HE  configuration  to  simulate  this 
effect  must  be  based  upon  numerical  calculations,  one  can  speculate  that 
this  could  be  accomplished  by  prechambering  part  of  the  HE  in  the  bottom 
part  of  the  water  stem. 

(c)  Air  Stemming.  There  is  limited  evidence  to  indicate  that  the  absenei 
of  stemming  in  HE  detonations  does  give  rise  to  a  degradation  in  cratering 


efficiency1 


Once  initiated  by  toe  HE  or  nuclear  detonation,  the  shock 
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propagates  radially  outward  not  feeling  the  presence  of  the  unstemmed 
hole  except  In  the  direction  along  the  axle  of  the  hole.  For  this  reason, 
the  spallation  mechanism  set  Into  motion  by  the  shock  wave  should  be  rel¬ 
atively  unaffected  by  the  lack  of  stemming.  However,  all  effects  which 
depend  upon  the  cavity  gas  may  be  greatly  affected. 

In  the  simplest  approximation,  the  borehole  can  be  considered  to  be 
a  high*  pres  sure  shock  tube.  The  application  of  Ideal  shock  tube  theory^ 
to  the  nuclear  case  indicates  that  the  Initial  velocity  of  the  cavlty-gas/alr 
interface  in  the  borehole  may  be  greater  than  20  meters  per  millisecond. 
(During  the  Marvel  experiment,  a  velocity  of  about  100  m/ms  was  ob¬ 
served^.  )  Similar  calculations  Indicate  that  a  nitromethane  high  explo¬ 
sive  may  exhibit  a  velocity  of  approximately  10  m/ms.  At  these  speeds, 
the  Interface  will  outdistance  the  shock  wave  in  the  adjacent  material, 
originating  a  secondary  shock  wave  from  the  interior  of  the  borehole. 
Should  such  interface  velocities  be  sustained,  then  a  significant  mass  of 
the  cavity  gas  will  be  ejected  up  the  borehole,  the  rate  depending  upon  the 
hole  diameter.  However,  such  rates  cannot  be  sustained  because  a  num¬ 
ber  of  effects  arise  to  oppose  them.  First,  as  the  interface  rises,  the 
gas  volume  increases,  decreasing  the  pressure  which  drivSs  the  gas  flow. 
Energy  losses  which  are  somewhat  localized  in  the  stem  region  occur 
through  the  generation  of  a  secondary  shock  wave,  fluid^friction,  and  heat 
transfer  to  the  stem  walls.  Ihis  'also  effects  a  pressure  decrease.  The 
principle  effect  is  probably  the  ablation  of  mats  from  the  stem  walls  which 
increases  the  mass  of  the  flowing  mixture  with  a  resulting  decrease  in 
velocity.  If  the  borehole  diameter  is  relatively  small,  the  stresses  pro¬ 
duced  by  the  shock  wave  in  the  material  surrounding  the  hole  may  be 
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sufficient  to  cause  closure  following  the  brief  venting  of  gee  resulting  » 

from  the  initially  high  interface  velocities.  | 

High-explosive  simulation  of  an  unstemmed  nuclear  cratering  shot  Is  j 

complicated  by  the  large  difference  In  the  initial  interface  velocities  in- 

| 

duced  by  the  differences  in  initial  pressures.  Two  approaches  appear  to  .1 

be  possible.  The  first  involves  simulating  the  hole  of  a  nuclear  shot  with 
a  larger  HE  hole.  The  choice  of  hole  diameters  would  be  based  upon  re¬ 
quiring  either  similar  mass  flows  up  the  stem  or  similar  cavity  pressure 
histories.  The  second  approach  involves  selecting  an  initial  HE  configure-  1 

tion  which  is  similar  to  the  shape  of  a  nuclear  cavity  (including  the  stem) 
at  some  late  time  when  the  pressure  and  interface  velocity  have  dropped 
to  levels  attainable  with  high  explosives.  Both  approaches  require  ex¬ 
tensive  numerical  modeling  and  detailed  knowledge  of  all  material  prop- 
erites  affecting  the  gas  flow.  Although  it  will  not  be  known  with  any  degree 
of  certainty  until  these  calculations  have  been  carried  out,  it  appears  that 
the  above  simulation  approaches  may  not  be  in  conflict  with  the  previously 
stated  criteria  for  fully-stemmed  detonations. 

(12)  Summary 

The  effects  discussed  in  the  preceding  paragraphs  and  the  resulting 

i 

similitude  criteria  are  listed  in  Table  1.  In  the  fully- stemmed  configura¬ 
tion,  most  of  the  dynamic  cratering  effedts  can  be  simulated  by  matching 
the  mound  kinetic  energies  (residual  particle  velocities)  and/or  the  low- 
pressure  cavity  gas  adiabats.  These  criteria  can  be  met  individually 
through  the  judicious  selection  of  the  high  explosive,  the  yield,  and  the 
charge  configuration.  Whether  the  criteria  can  be  collectively  met  is  at 
present  unknown. 
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TABLE  1 


SUMMARY  OF  SIMILITUDE  REQUIREMENTS: 


EFFECT 

1.  Early- time  energy  source 

behavior 

2.  Spallation  mechanism 

3.  Ground-shock-induced  airblast 

4.  Seismic  effects 

5.  Gas -acceleration  mechanism 

6.  Depth  of  burst  and  crater 

dimensions 

7.  Missile  ranges 

8.  Gas-vent-induced  airblast 

9.  Venting  of  cavity  gas  and 

radioactivity 

Violent  venting 
Nonviolent  venting 

10.  Cloud  formation 

Base  surge 
Main  cloud 

11.  Effects  of  stemming 

Stemmed  case 
Water- stemmed  case 

Unstemmed  case 


SIMILITUDE  CRITERION 
None 


Residual  mound  velocities 

Peak  surface  spall  velocity 

Residual  mound  velocities 

Low-pressure  adiabat 

Residual  mound  velocities  and  Low- 
pressure  adiabat 

Peak  surface  velocity 

Low-pressure  adiabat 

Low-pressure  adiabat 


Particulate  tracer 
?  ? 


Residual  mound  velocities 
Low-pressure  adiabat  (?) 


Above  criteria 
Cavity. gas /steam  pressure 
profile  (? ) 

Gas  pressure  and/or  velocity 
profile  (? ) 


INDIVIDUAL  SIMILITUDE  CRITERIA  CAN  BE  MET  THROUGH  THE 
JUDICIOUS  SELECTION  OF  THE  HIGH  EXPLOSIVE,  THE  YIELD,  AND 
THE  CHARGE  CONFIGURATION 
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III.  PROJECT  DIAMOND  ORE 

(1)  General  Description 

Project  DIAMOND  ORE  consist!  of  a  series  of  high-explosive  field 
experiments  and  associated  theoretical  and  experimental  studies  directed 
toward  (1)  investigating  the  role  of  stemming  materials  in  both  nuclear 
and  high-explosive  cratering  shots  at  shallow  and  optimum  depths,  and 
i'  (2)  arriving  at  specific  design  criteria  for  high-explosive  simulation  of 
nuclear  detonations.  Although  existing  numerical  codes  have  the  capa¬ 
bility  of  treating  nuclear  and  high-explosive  configurations  such  as  those 
of  Interest  here,  the  codes  employ  many  assumptions  about  the  behavior  of 
matter  in  and  about  the  explosive  environment.  The  codes  also  require, 
as  input  data,  very  detailed  descriptions  termed  "equations  of  state"  (EOS) 
of  the  energy  source  and  the  cratering  medium .  Consequently,  great 
credibility  cannot  be  attached  to  design  specifications  based  on  theoreti¬ 
cal  calculation*  alone.  Cn  the  other  hand,  although  considerable  experi¬ 
ence  has  been  gained  in  the  field  of  HE  cratering,  there  Is  no  means  by 
which  this  can  be  directly  related  to  nuclear  cratering.  The  intent  of 
DIAMOND  ORE  Is  to  mesh  cratering  theory  and  experiment  together  by 
providing  a  scries  of  well-documented  experiments  by  which  the  codes  can 
be  tested  thereby  enhancing  the  credibility  of  the  computer-generated 
criteria  for  high-explosive/nuclear  equivalence. 

As  presently  envisioned,  DIAMOND  ORE  is  to  be  conducted  in  three 
phases.  Figures  i  and  4  are  flow  charts  which  depict  the  interactions 
between  tho  various  parts  of  the  shallow  and  optimum  depth-of-burst  programs 
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Figure  3 

Diamond  Ore  Program  at  Optimum 

Craterinq  Depth. 


EFFECTS 


Figure  4 

Diamond  Ore  Program  at  Shallow 
(6  meter}  Depth. 


Phil  ho  I  iw  a  continuing.'  study  of  hif»h  explosives  characteristics,  cra¬ 
tering  media  properties,  and  fallout  simulants  and  has  boon  in  progress 
since  late  1970.  The  fallout  simulant  program  arising  from  the  Phase  I 
studies  will  be  discussed  in  detail  in  a  later  section  of  this  paper.  The 
site  of  the  Phase  II  experiments  coincides  with  that  of  the  Pre-GONDOLA 
series  of  1966-68.  Numerical  simulation  of  the  Pre-GONDOLA  I  events 
has  been  coupled  with  the  known  experimental  properties  of  the  clay  shkle 
at  the  site  to  arrive  at  a  consistent  material  equation  of  state1  J.  Simi¬ 
lar  equation-of- state  studies  are  now  being  carried  out  on  the  aluminized 
ammonium  nitrate  slurry  which  is  the  explosive  selected  for  the  Phase  II-A 
series. 

Phase  II  includes  two  series  of  cratering  detonations  to  be  fielded  at 
the  Lewis  Reservoir  near  Fort  Peck,  Montana.  The  first  series,  Phase  II-A, 
is  to  be  fielded  in  October  1971  and  consists  of  a  stemmed  detonation  and 
an  unstemmed  detonation  at  near-optimum  depth  of  burst,  and  a  stemmed 
detonation  at  about  half-optimum  depth.  The  yields  are  identical  and  are 
in  the  low  sub-kiloton  range.  The  results  of  these  experiments  will  pro¬ 
vide  direct  comparisons  of  the  effects  of  stemming  and  depth  of  burst. 

The  instrumentation  will  be  such  as  to  provide  data  on  the  behavior  of  the 
shale  and  the  slurry  under  conditions  not  previously  investigated.  Phase  II-A 
is  then  intended  to  serve  as  a  well-instrumented  test  series  upon  which  the 
hydrodynamic  computer  codes  can  be  tested,  and  the  explosive  and  shale 
equations  of  state  refined.  When  the  theoretical  program  has  progressed 
to  the  point  that  the  Phase  II-A  series  can  be  correctly  simulated  numeri¬ 
cally,  then  a  thorough  knowledge  of  the  shale  and  slurry  equations  of  state 
and  the  dynamic  effects  of  stemming  has  been  gained. 
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Willi  thu  information  gained  from  Phase  1I-A  and  past  oxpuritsncu  in 
applying  hydrodynamic  codes  Lo  nuclear  detonation*,  it  will  be  possible  to 
predict  the  effect*  of  nuclear  detonation*  in  the  Fort  Puck  shale  in  the 
following  configuration*:  unstemmed,  water-stemmed,  and  fully-stemmed 
at  optimum  depth  of  burst,  and  unstemmed  at  half-optimum  depth.  Taking 
into  account  the  qualitative  simulation  criteria  discussed  in  Section  Q  of 
this  paper,  various  high -explosive  emplacement  configurations  can  be 
studied  to  determine  wV&c)i  will  best  model  the  nuclear  explosive  and  its 
stemming.  This  will  be  an  iterative  process  which  will  converge  upon  tha 
explosive  designs  to  be  used  in  the  Phase  II-B  experiments.  If,  at  this 
stage  of  the  program,  it  should  be  determined  that  the  slurry  explosive 
is  incompatible  with  similitude  requirements,  the  use  of  a  more  appro¬ 
priate  explosive  will  be  considered  at  this  time.  The  Phase  Il-B  series 
is  scheduled  for  summer  of  1972  and  is  intended  to  be  an  experimental 
verification  of  the  theoretical  explosive  designs. 

It  is  anticipated  that  the  Phase  II  series  will  provide  information  on 
the  relative  importance  of  the  various  material  properties  as  related  to 
stemming,  depth,  and  yield.  The  design  techniques  employed  in  Phase  II-B 
can  then  be  adapted  to  designing  additional  simulation  experiments  in  other 
geologic  materials.  This  test  series  called  Phase  III  would  be  directed 
toward  demonstrating  the  feasibility  of  HE  simulation  of  nuclear  explosions. 

The  DIAMOND  ORE  objectives  are  summarized  in  Table  2. 

(2)  Site  Selection 

The  selection  of  Fort  Peck,  Montana,  as  the  site  of  the  Phase  II  ex¬ 
periments  was  based  upon  several  considerations.  First,  the  clay  shale 
in  which  the  series  will  be  conducted  is  of  interest  in  civil  and  military 


TABLE  2 


PROJECT  DIAMOND  ORE  OBJECTIVES: 


Design  and  execute  high-oxploalva  experiment*  which  will  best  sim¬ 
ulate  nuclear  cratering  detonation*  under  various  stemming,  depth,  and 
material  conditions. 


PHASE  1  Study  cratering  media  equations  of  state 
Study  high- explosive  equations  of  state 
Select  high  explosive  for  simulation 
Devise  fallout  simulant  program 


PHASE  1IA  Provide  direct  comparisons  of  effects  of  stemming  and 
depth  of  burst 

Provide  data  on  behavior  of  clay  shale  and  slurry  under 
conditions  not  previously  investigated 
Servo  as  a  well-documented  experiment  upon  which  the 
hydrodynamic  codes  can  be  tested  and  refined 


PHASE  IIB  Provide  additional  data  as  required 

Serve  as  experimental  verification  of  theoretical  explosive 
designs  in  clay  shale 


PHASE  III  Demonstrate  feasibility  of  high-explosive  simulation  of 
nuclear  cratering  detonations  in  varied  geologic  media 


Applications.  Socondly,  the  site  Is  free  of  abundant  wildlife  and  distance 
from  large  population  centers,  thereby  reducing  the  environmental  impact 
of  the  detonations .  The  site  is  sufficiently  large  and  free  of  mountainous 
terrain  to  allow  the  implementation  of  a  fallout  simulation  and  collection 
program.  The  area  has  been  the  site  of  several  other  high->explosive  ex¬ 
periments,  and  the  experience  in  conducting  flold  operations  there  tended 
to  reduce  project  costs.  However,  the  most  important  consideration 
was  the  existence  of  extensive  data  on  the  material  properties  of  clay  shale, 
thereby  reducing  data  acquisition  costs. 

(3)  Explosive  Selection 

In  fielding  an  experimental  cratering  program  as  extensive  as 
DIAMOND  ORE,  the  cost  of  explosives  becomes  a  primary  consideration. 
Past  experience  in  the  field  of  explosive  excavation  has  shown  that,  in 
terms  of  overall  cratering  efficiency,  the  slurry  explosives  are  by  far 
the  most  economical.  Because  slurry  can  be  emplaced  In  liquid  form  and 
is  classed  as  a  blasting  agent  rather  than  a  high  explosive,  coots  of  hand¬ 
ling  are  also  reduced. 

As  will  be  discussed  later,  the  use  of  a  fallout  simulant  requires  that 
the  simulant  particles  be  uniformly  distributed  throughout  the  HE. 

Because  the  slurry  can  be  made  to  gel  after  emplacement,  it  is 
ideally  suited  to  this  purpose.  The  simulant  particles  can  be  uniformly 
mixed  while  the  slurry  is  liquid  and  will  remain  so  after  emplacement. 

Finally,  some  slurry  formulations  provide  very  high  energy  release 
per  initial  unit  volume,  thereby  more  closely  approximating  the  initial 
nuclear  cavity  than  less  energetic  explosives.  Unfortunately,  the  slurry 
equation  of  state  is  presently  unknown,  and  the  more  precise  similitude 
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requirements  discussed  In  Section  11  of  this  paper  therefore  cannot  be 
applied  to  it.  In  view  of  its  other  desirable  properties  and  the  fact  that 
it  is  of  considerable  interest  in  its  own  right,  an  attempt  to  make  use 
of  slurry  as  a  nuclear  explosive  simulant  was  felt  to  be  justified.  Accord- 
ingly,  slurry  was  selected  for  the  Phase  11-A  experiments;  and  determining 
its  equation  of  state  was  made  one  of  tho  principal  objectives  of  Phase  1. 
However,  should  the  slurry  prove  unacceptable  for  simulation  purposes, 
concurrent  studies  are  being  conducted  to  select  an  alternative. 

(4)  Phase  II-A  Technical  Programs 

The  following  outlines  the  technical  programs  to  be  executed  in  con¬ 
junction  with  Phase  1X-A. 

(a)  Site  Investigation.  Prior  to  the  final  selection  of  the  ground  zeros 
for  the  three  Phase  H-A  events,  seismic  refraction  surveys  and  continu¬ 
ous  borehole  loggings  were  conducted  at  a  number  of  proposed  sites. 

These  investigations  provide  direct  measurements  of  sonic  velocities  and 
indications  of  anisotropies  in  the  shale. 

(b)  Direct  Measurements  of  Material  Properties.  Core  samples 
have  been  provided  to  the  U.  S.  Army  Engineer  District,  Missouri  River 
Division,  for  measurements  of  unconfined  compressive  strengths,  residual 
shear  strength,  Young's  modulus,  Attenberg  limits,  and  other  material 

3 

properties.  Samples  have  also  been  provided  to  L  for  plane  strain  mea¬ 
surements  which  will  yield  data  on  the  static  elastic  moduli  and  material 
strength. 

(c)  Close-In  Ground  Motion  and  Earth  Stress  Measurements.  (Figure  5) 
For  the  two  optimum  DOB  detonations,  acceleration,  velocity,  and  stress 
gauges  will  be  employed  in  a  linear  array  of  boreholes  to  provide  the 
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motion  and  stress  time  histories  of  the  medium  under  the  influence  of 

3  3 

the  shock  front  for  use  in  L  and  S  postshot  calculations.  It  is 
anticipated  that  the  radial  and  tangential  gauges  in  holes  B,  C,  and  D 
may  provide  data  for  determining  dynamic  values  for  the  elastic  moduli. 

(d)  Surface  Motion  Measurements  (Figure  5).  Early  time  high-speed 
photography  (1000  FPS)  will  record  each  detonation  for  the  study  of  veloc¬ 
ities  and  the  acceleration  of  particles  within  the  crater  area,  and  for  the 
study  of  mound  growth  and  venting.  Accelerometers  and  velocity  gauges 
will  also  be  used  to  measure  the  rate  of  mound  growth. 

(«)  Stemming  Pressure  Measurements  (Figure  5).  A  vertical  array 
of  stress  gauges  will  be  emplaced  within  the  air  column  of  the  unstemmed 
shot.  An  additional  array  will  be  located  just  outside  the  unstemmed  hole. 
Both  arrays  will  provide  stress  and  time-of-ar rival  measurements  re¬ 
quired  for  code  calculations. 

(f)  Uniformity  of  Detonation  Measurements  (Figure  5).  The  detona¬ 
tion  history  of  the  explosive  detonation  v/ill  be  recorded  by  emplacing  four 
"rate  sticks"  within  the  explosive  material.  These  measurements  are  re¬ 
quired  to  verify  the  detonation  uniformity  of  the  slurry  explosive  and  will 
provide  detonation  velocity  and  perhaps  shock  velocity  data  for  the  slurry. 

(g)  Crater  Measurements.  The  dimensions  of  all  craters  will  be 
measured  by  conventional  ground  survey  to  provide  preliminary  crater 
dimensions.  Aerial  mapping  will  provide  precise  dimensions  of  experi¬ 
mental  results.  Aerial  photography  of  the  craters  will  be  taken  after  all 
three  detonations  have  been  conducted. 

(h)  Ejecta  Analysis .  Missile  ranges  and  directions  relative  to  each 
ground  zero  will  be  determined  by  conven  ional  ground  survey  for  each 
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detonation.  This  data  will  be  used  to  relate  missile  areal  densities  to 
missile  ranges  in  the  area  between  the  limits  of  the  continuous  ejecta  and 
the  maximum  missile  range.  Areal  distribution  of  the,  continuous  ejecta 
will  be  determined  by  the  "point  count  technique".  Limits  of  the  continu¬ 
ous  cjccla  and  maximum  missile  range  will  be  marked  in  a  manner  which 
will  be  dotoctable  on  the  postshot  aerial  photography. 

(i)  Technical  Photography.  In  addition  to  the  photography  mentioned 
above,  documentary  16mm  photography  will  record  the  cloud  formation 
for  each  event,  and  incidental  still  photographs  will  be  taken  of  each  deto¬ 
nation.  Although  no  cloud  studies  are  planned  at  this  time,  the  Information 
will  be  available  should  such  studies  be  desired  at  a  later  date. 

(j)  Fallout  Collection.  Fifty  (50)  collection  trays  will  be  placed  in 
the  expected  downwind  area  on  all  three  detonations  in  order  to  provide 
basic  data  as  to  fallout  mass  deposition  relative  to  ranges  from  Ground 
Zero.  This  data  is  desired  for  the  fallout  collection  array  designs  to  be 
used  in  Phase  II-B.  Samples  from  the  fallout  trays  will  be  taken  for  use 
in  tracer  background  studies.  The  location  of  these  trays  relative  to  the 
wind  direction  at  shot  time  will  not  be  a  controlling  factor  in  the  firing 
decision.  Although  no  fallout  simulation  is  scheduled  for  Phase  II-A,  un¬ 
coated  silica  particles  will  be  mixed  with  the  explosive. 

(k)  Meteorological  Program.  Wind  speed  and  direction  will  be  de¬ 
termined  three  times  a  day  (at  0900,  1200,  and  1500  hrs)  at  the  ground 
surface  and  at  altitudes  of  100,  200,  300,  400,  500,  600,  800,  1000,  1500, 
2000,  and  2500  meters.  This  data  collection  will  be  initiated  five  days 
prior  to  shot  date  for  Event  II-A- 1  (unstemmed)  on  a  daily  basis. 

In  addition,  on  shot  days,  wind  data  collection  will  be  initiated  at  the  time 
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(l)  Clono-in  Airblasl.  Peak  overpressures  within  the  range  of  0. 03  J 

to  1.0  pal  will  be  measured  on  all  three  shots  at  ranges  of  250',  500',  ; 

1000',  and  2000'.  Additional  gauges  will  be  at  4000*  on  the  unstemmed  shot 

i 

shot  and  at  4000'  and  10000'  oh  the  half-optimum  depth  shot. 

(m)  Seismic  Investigation.  Two  ground- shock  stations  will  be  em- 
ployed  to  measure  particle  velocity  for  comparative  soismic  measure¬ 
ments  using  EERO  in-house  capability.  Gauge  locations  will  be  field  de¬ 
termined  and  surveyed  after  emplacement. 
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IV.  VENTED  RADIOACTIVITY  AND  FALLOUT  DISTRIBUTION 

To  determine  the  effect  the  type  and  degree  of  stemming  will  have 
on  the  fraction  of  material  vented  from  the  simulated  nuclear  crater  and 
its  distribution  around  the  shot  point,  tagged  mineral  particles  are  being 
employed  which  can  be  identified  in  the  post- explosion  debris.  The  parti¬ 
cles  are  of  a  size  and  density  such  that, while  in  the  debris  cloud,  they 
will  respond  to  the  influences  of  winds  and  gravity  as  do  the  same  size 
radioactive  particles  from  a  nuclear  explosion.  Thus,  if  the  venting 
process  of  a  nuclear  explosive  is  simulated,  particle  areal  deposition 
should  bo  duplicated  for  a  similar  wind  condition.  The  nuclide  composi¬ 
tion  of  a  particle  and  the  resulting  fallout  field  dose  rate  will  still  require 
generation  by  numerical  techniques. 

(1)  Previous  Fallout  Simulation  Experiments 

While  the  use  of  tracers  for  this  purpose  is  not  new,  the  approach 

being  taken  is  new.  In  previous  experiments,  Project  YO-YO  and 

108  76 

Pre-BUCGY,  radioactive  tracers  (  Au  and  As  in  Project  YO-YO 
140 

and  La  in  Pre-BUGGY)  were  employed.  Conclusions  from  these  ex¬ 
periments  were  that:  (1)  The  fraction  of  the  radioactive  material  that  es¬ 
caped  was  related  to  depth  of  burst  and  apparent  crater  radius,  but  the 
escape  fractions  were  higher  than  those  expected  from  nuclear  detonations 
at  the  same  ratio  of  depth  of  burst  and  apparent  crater  radius;  and  (2) 
radioactive  tracers  in  chemical  explosives  did  not  simulate  fallout. 

Examination  of  these  experiments  indicates  that  the  method  of  place¬ 
ment  of  the  radioactive  tracer  in  the  chemical  explosive  may  have  lead 
to  the  relatively  high  amount  of  radioactivity  vented.  In  both  experiments, 
the  radioactivity  was  placed  in  a  capsule  located  in  the  center  of  the  charge. 
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Upon  damnation  there  was  no  assurance  that,  prior  to  venting,  the  radio¬ 
active  tracer  would  uniformly  mix  with  the  explosive  gases  or  would  not 
be  preferentially  vented  in  a  gaseous  jet. 

Failure  of  the  radioactive  tracer  to  simulate  fallout  is  attributed  to 
tlio  differences  in  the  temperature,  pressure  and  venting  processes  he- 
tween  nuclear  and  chemical  explosives.  As  was  previously  noted,  the 
venting  process  is  strongly  influenced  by  the  cavity  pressure  and  temper¬ 
ature  history  and.  in  turn,  the  fraction  of  volatile,  refractory  and  inter¬ 
mediate  radionuclides  vented.  Summarised,  it  is  known  that  if  the  temper 
ature  and  pressure  at  vent  time  are  higher,  a  more  violent  mixing  of 
radionuclides  will  result  and  the  vented  material  will  contain  a  higher 
fraction  of  refractory  and  intermediate  radionuclides.  Distribution  of 
the  refractory  and  volatile  radionuclides  differs  among  fallout  particle 
sizes  with  the  cavity  pressure  and  temperature  at  vent  time.  When  pres¬ 
sures  and  temperatures  are  low,  refractory  nuclides  are  deposited  on 
the  larger  particles  of  overburden  and  fall  close  to  ground  zero.  When 
pressures  and  temperatures  are  high,  refractory  elements  are  associated 
with  particles  having  a  smaller  mean  radius  and  are  more  abundant  in  the 
nuclear  cloud.  Then,  refractory  radionuclides  are  found  to  be  volume 

contained  while  volatile  elements  are  surface  deposited. 

roi 

Data  from  the  Pre-BUGGY  experiment1  J  showed  that  the  radioactive 
140 

tracer  employed  (  La)  was  surface  adsorbed  on  the  particles  in  the 
detonation  cloud.  For  Pre-BUGGY,  96%  of  the  recovered  radioactivity 
was  found  on  small  particles  which  constituted  only  8%  of  the  particle 
mass  but  an  estimated  90%  of  the  surface  area  of  the  deposited  particles. 
With  the  majority  of  tracer  activity  associated  with  the  smaller  particle 
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sizes,  \y intis  acting  on  the  explosion-produced  cloud  carried  the  small 
particles  to  great  distances  from  surface  aero  and  distorted  the  fallout 
patterns. 

To  circumvent  the  problems  of  previous  experiments,  the  approach 
being  pursued  to  measure  the  vent  fraction  consists  of  using  a  neutron  acti- 
vable  chemical  element  surface  adsorbed  on  quarts  particles  of  known  size  and 
density.  These  tagged  particles  are  uniformly  mixed  throughout  the  ex¬ 
plosive  charge.  The  rationale  for  this  approach  is  discussed  in  the  follow¬ 
ing  paragraphs. 

(2)  Tracer  Selection 

Since  the  test  program  is  to  be  conducted  in  a  number  of  locations 
and  geologic  medium,  an  inert  tracer,  suitable  for  neutron  activation 
after  collection,  is  being  employed  to  avoid  the  problems  associated  with 
the  use  of  radioactive  materials.  Use  of  neutron  activable  tracers  rather 
than  a  radioisotope  eliminates: 

(a)  The  need  for  an  AEC  or  State  license  for  use  of  radioactive 
materials . 

(b)  The  logistic  problems  of  providing  sufficient  tracer  for  a  field 
test  that  iB  weather  dependent. 

(c)  The  requirement  to  have  field  equipment  to  measure  radioactivity 
levels . 

(d)  The  difficulties  of  conve&ting  radiation  measurements  to  mean¬ 
ingful  fallout  values, 

(c)  The  radiation  control  measures  required  during  handling  of  the 
tracer  and  placement  in  the  explosive  charge. 

(f)  The  problems  associated  with  releasing  radioactivity  to  the 
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The  neutron  actlvablo  tracer  coated  on  particles  of  known  size  and 
density  is  being  mixed  with  the  explosive  to  assure  a  uniform  mixture 
in  the  gases  resulting  from  the  explosion  and  to  prevent  preferential 
venting  of  the  tracer. 

The  coated  particles  range  in  size  from  125  to  175  microns  and  are 
a  pure  quartz.  (Wedron  aand)  with  a  particle  density  of  2.  65  grams /cc. 

This  size  range  was  selected  because  it  comprises  an  important  size 
fraction  in  local  fallout  and  falls  to  earth  within  a  reasonable  distance 
from  ground  zero.  This  limited  downwind  fallout  area  permitB  an  exten¬ 
sive  fallout  sample  collection  effort  to  be  fielded.  This  particle  range 
also  brackets  the  143>micron  particle  size  assumed  to  be  nominal  in  cer¬ 
tain  fallout  prediction  schemes.  Quartz  particles  are  being  employed 
since  quartz  has  a  high  melting  point  (1600°C)  and  will  survive  the  high 
explosive  environment  without  melting.  In  addition,  when  irradiated, 
pure  quartz  produces  no  radionuclides  which  interfere  with  the  detection 
of  the  tracer  selected. 

Iridium  is  the  neutron  activable  element  of  choice  for  coating  the 
particles.  This  element  was  selected  after  reviewing  the  litcrature^^'  ^ 
pertaining  to  the  elemental  constituents  of  various  geologic  n.cdia,  investi¬ 
gating  the  physical  properties  of  various  chemical  elements  suitable  for 
neutron  activation,  and  examination  of  the  gamma  ray  spectra  of  geologic 
media  from  proposed  test  site  locations.  The  literature  review  indicated 
that  the  natural  abundance  of  iridium  in  geologic  media  was  about  1  x  10""^ 
This  low  abundance  in  nature  minimizes  the  interference  with  detection 
of  traced  particles  from  natura.  y  occurring  iridium.  Iridium's  melting 
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point  La  approximately  2400°C.  This  high  melting  point  minimizes  the 

loss  u£  iridium  from  the  coated  particles  due  to  boiling.  The  explosive 

temperatures  are  estimated  to  be  about  2500°C.  The  thermal  neutron 
191 

cross  section  of  Ir  which  has  a  natural  abundance  of  37.3%  is  750  barns. 

192 

Its  activation  product  (  ;  Ir)  has  a  half-life  of  74  days  and  undergoes  beta 
192 

decay  to  Pt,  emitting  316  kev  gamma  photons  with  a  branching  intensity 
of  85%  and  468  kev  photons  with  50%.  The  high  cross  section  permits 
activation  in  relatively  short  periods  in  the  reactor  (1  hour  at  a  neutron 
flux  of  5  x  10  n/ cm  / s)  and  the  high  abundance  of  photons  permits  easy 

detection.  Spectra  of  various  irradiated  media  were  examined  In  a  3"  x  3" 
sodium  iodide  crystal  and  no  naturally  occurring  iridium  of  significance 
was  found  to  be  present.  After  20  days,  the  gamma  photons  present  from 
other  long-lived  radionuclides  present  only  minimal  interference  with  the 
detection  and  quantification  of  the  Iridium.  Figure  6  shows  the  spectra 
of  an  irradiated  medium  from  a  site.  In  the  figure  line  2  is  the  natural 
spectra,  of  the  soil;  line  3  shows  the  iridium  spectra;  and,  line  1  shows 
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the  spectra  of  10  grams  of  iridium  (approximately  25  coated  particles) 

c 

in  one  gram  (approximately  2  x  10  particles)  of  natural  soil  after  sieving 

to  obtain  the  125  to  175  micron  particles. 

(3  )  Preliminary  Simulation  Tests 

In  addition  to  the  studies  indicated  above,  a  series  of  tests  have  been 
conducted  to  determine  if  the  iridium- coated  quartz  particles  will  survive 
an  explosive  environment.  In  one  set  of  tests,  0.4  pounds  of  iridium- 
coated  quartz  particles  were  uniformly  mixed  with  4  pounds  of  C-4  explo¬ 
sive.  These  explosive  charges  were  detonated  2.4-feet  below  the  ground 
surface  and  "fallout"  samples  collected  and  sieved  into  throe  fractions: 


(1)  lean  than  125  microns,  (2)  125  to  175  microns,  and  (3)  175  to  350  mi 
microns.  Analysis  of  the  collected  samplos  indicated  that  the  iridium 
did  not  appreciably  migrate  from  the  125  to  175  micron  range. 

In  another  set  of  3  tests,  0.  8  pounds  iridium-coated  particles  were 
uniformly  mixed  with  8  pounds  of  alluminized  ammonium  nitrate  slurry 
called  TD-2  and  manufactured  by  the  IRECO  Company  of  Salt  Lake  City, 
Utah.  The  results  of  the  three  tests  were  identical  and  indicated  that 
the  fallout  could  be  collected  and  the  iridium  content  or  particle  content 
could  be  determined  after  irradiation. 

Cratering  tests  performed  with  TD-2  explosives  containing  various 
weights  of  sand  per  weight  of  explosives  indicated  that  explosive  perform¬ 
ance,  for  cratering  purposes,  was  not  seriously  degraded  if  the  amount 
of  sand  added  to  a  charge  was  not  more  than  10%  of  the  charge  weight. 

To  assist  in  the  transition  from  pound-charge  tests  to  multi-ton 
charge  field  events,  a  series  of  four  1-ton  charges  containing  10%  iridium- 
coated  sand  was  executed  at  Trinidad,  Colorado,  during  the  period  9  to 
25  September  1971.  All  four  charges  were  emplaced  in  a  massive 
sandstone  medium  at  a  constant  depth  of  burial,  20  feet,  but  with  varying 

stemming  conditions:  full  stemming,  water  stemming,  no  stemming  in  a 
36-inch  diameter  hole;  and  no  stemming  in  a  4-inch  diameter  hole. 

(4)  Tracer  Collection 

To  determine  the  amount  of  material  vented  from  each  of  the  Trinidad 
tests,  approximately  200  debris  samples  were  collected  about  each  test. 
Debris  was  collected  in  24"  x  24"  x  2"  aluminum  trays.  Each  tray  con¬ 
tained  louvers  spaced  one  inch  apart  and  slanted  at  45  degrees  to  the 
horizontal  to  insure  that  particles  arriving  at  a  tray  did  not  bounce  or 
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blow  out  of  the  tray  collector.  Figures  7  and  8  show  the  "fallout"  collector 
array  which  extended  out  to  2000  yards  from  the  surface  aero  of  each 
detonation. 

(5)  Selection  of  Tray  Locations 

Selection  of  collector  tray  locations  was  based  on  estimates  of: 

(1)  the  limit  of  continuous  ejecta.  (2)  a  mass  deposition  equation  developed 

3 

by  M.  Nordyke  of  L  (unpublished),  (3)  fall  times  of  tagged  particles 
from  the  top  of  the  cloud,  (4)  detectability  of  mass  deposition  measure¬ 
ments,  and  (5)  cloud  data  from  previous  1-ton  detonations  conducted  at 
the  Trinidad  Test  Site. 

Nordyke's  equation  is: 


S  x  10l 


1  +  0.02  vJ 


(ifa) 


(3.85  ±  0.  1  v) 


2 

where  8  -  mass  deposition  (g m/m  ), 

W  =  yield  of  the  detonation  (kt), 
v  s  wind  speed  (mph), 

R  =  distance  from  SGZ  (10  yds), 

The  plus  in  th<-  exponent  is  for  upwind  and  crosswind  predictions,  while 
the  minus  is  for  downwind  predictions. 

There  are  several  restrictions  for  the  use  of  the  equation,  some  of 
the  major  ones  are: 

4 

a.  for  downwind  and  crosswind  predictions,  10  <18  <10  ; 

2  4 

b.  for  upwind  predictions,  10  £  8  <  10 

c.  v  £20  mph. 

Figure  9  shows  the  predicted  mass  deposition  versus  downwind  distance 
for  the  1-ton  Trinidad  detonations  and  indicates  the  apparent  crater  radius 


? 
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Figure  7 

Intermediate  Range  Collector 
Array  for  Project  Trinidad. 
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The  detonation  times  were  selected  so  as  to  constrain  the  wind  to  a  ' 

speed  of  5  to  15  miles  per  hour  centered  along  the  90  degree  axis  of  the 

I 

array.  Cloud  height,  growth,  and  travel  photography  and  wind  velocity 
measurements  at  each  100  ft  up  to  2000  ft  were  taken  to  provide  data  for 
the  subsequent  analysis  to  be  performed. 

(6)  Data  Analysis 

The  collected  material  from  each  tray  was  weighted  to  determine  the 

mass  deposition  per  unit  area  and  then  sieved  into  size  fractions!  88  to 

125  microns,  125  to  175  microns,  and  175  to  350  microns  for  irradiation. 

These  three  samples  from  each  collection  point  will  be  encapsulated, 

irradiated  with  thermal  neutrons,  and  the  number  of  tagged  particles 

19? 

determined  by  measuring  the  Ir  content.  The  size  fractions  over  and 
under  the  tagged  particle  size  will  also  be  processed  to  verify  that  agglom¬ 
eration  or  fracturing  of  the  particles  did  not  occur,  and  that  the  iridium 
did  not  migrate. 

1 92 

Data  from  the  mass  deposition,  sieve  fractions,  and  Ir  content 
will  permit  an  integration  over  the  area  in  the  fallout  pattern  to  compute 
a  fraction  vented  for  each  of  the  events. 

The  fallout  pattern  from  the  fully-stemmed  high- explosive  detonation 
will  be  converted  to  radiation  dose-rate  contours  by  relating  to  fully- 
stemmed  nuclear  detonation  data.  Data  from  the  4-inch  and  36-inch  un¬ 
stemmed  and  the  water-stemmed  detonations  will  be  similarly  treated 
using  best  estimates  for  the  fractionation  effects  which  may  occur.  Frac¬ 
tionation  effects  for  these  cases  ars  the  subject  of  additional  theoretical 
study. 
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Figure  2 

Response  of  cratering  medium 
to  shock. 


Figure  3 

Diamond  Ore  Program  at  Optimum 
Cratering  Depth. 
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Figure  7 

Intermediate  Range  Collector 
Array  for  Project  TrlnluaJ. 
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MODIFIED  FACTORIAL  EXPERIMENTS 
FOR  ANALYSING  POISSON  DATA 


L.  Ott  and  W.  Mendenhall 
University  of  Florida 


I.  Introduction 


Recent  emphasis  on  research  In  the  areas  of  air  and  water 
pollution  has  focused  attention  on  the  need  for  better  methods 
of  analysing  Poisson  data.  We  refer  particularly  to  the  problem 
of  relating  mean  response  to  a  set  of  independent  variable 
for  purposes  of  estimation.  Thus  an  experimenter  may  wish  to 
estimate  the  mean  particle  count  per  unit  volume  of  air  at  a 
given  location  as  a  function  of  the  rates  of  input  of  pollutants, 
wind  velocity  and  direction.  Or,  he  might  wish  to  relate  the 
accident  count  per  unit  time  to  a  set  of  causative  or  related 
variables. 

Statistical  analyses  of  the  effect  of  a  set  of  independent 
variables  on  a  Poisaon  response  have  traditionally  relied  on  the 
use  of  a  transformation  on  the  response  data,  Thus  a  square 
toot  (or  modified  square  root)  transformation  is  employed  to 
stabilize  the  variance  of  the  response  prior  to  an  analysis  of 
variance,  thereby  satisfying  one  of  the  assumptions  implicit  in 
the  analysis  of  variance  F  teats.  This  method  of  analysis  is 
satisfactory  if  one  is  solely  interested  in  tests  of  hypotheses, 
but  it  is  less  than  satisfactory  if  estimation  is  the  goal.  In 
many  cases  the  experimenter  will  know  in  advance  that  the  Poisson 
mean  response  must  vary,  at  least  an  infinitesimal  amount,  as 
the  levels  of  the  independent  variables  are  changed.  Indeed,  he 
might  expect  the  change  in  mean  response  to  be  substantial.  When 
this  is  tha  case,  he  wishes  to  estimate  the  difference  in  mean 
response  for  particular  treatments  (factor  level  combinations)  or, 
when  one  or  more  of  the  independent  variables  are  quantitative, 
he  will  carry  the  estimation  process  further  and  fit  a  response 
curve  or  surface. 

Statistical  analysis  by  transformation  la  unsuitable  when 
estimation  la  tha  goal  because  one  is  estimating  the  expected 
value  of  the  transformed  data.  Thus  the  square  root  transforma¬ 
tion  of  a  Poisson  random  variable,  y,  leads  to  the  estimation  of 
E(/y).  Squaring  the  least  square  estimate  of  Etv'y)  does  not 
lead  to  the  "best"  estimate  of  the  Poisson  mean,  E{y).  We  demon¬ 
strate  this  fset  with  a  small  experiment. 


N  «*  1000  samples  of  size,  n  ■  10,  were  collected  on  a 
Poisson  random  variable,  y,  with  mean  m  ■  2.  Two  confidence 
Intervals  were  constructed  for  each  sample.  Interval  Estimation 
Method  IH  used  the  untransformed  data  with  the  intervals  computed 
according  to  the  formula, 

y  +  2/y/Io  ,  where  y  denotes  the  sample 

mean.  The  same  sample  response  measurements  were  used  for 
Interval  Estimation  Method  It 2  except  that  each  observation  was 
transformed  to  y*  where  y*  »  fy.  Then  the  traditional  least 
square  confidence  interval  was  computed  using 

y*  +  2s//l0 


where 


s  - 


(yj  -  y*)2 


Then  each  endpoint  of  the  interval  was  squared  to  obtain  a 
confidence  interval  on  E(y), 

A  comparinon  of  the  two  interval  estimation  methods  can  be 
made  by  analysing  the  confidence  intervals  generated  by  the  two 
methods  for  the  1000  samples  of  size  n  *  10.  Table  1  shows  the 
average  and  standard  deviation  of  the  width  of  the  intervals  for  the 
two  methods,  the  average  value  of  the  centerpoint  and  the  fraction 
ci!  times  the  intervals  enclosed  m. 


The  remainder  of  this  article  has  been  reproduced  photographically  from  the 
authors  manuscript. 
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Table  lj  A  Comparison  of  Confidence 
Intervals  for  m  Using  Transformed  and  Untransformed  Data 


Raw  Data 
Intervals 

Transformed 
Data  Intervals 

Average  width 

1.776 

1.948 

Standard  Deviation 
of  Widths 

0.200 

0.475 

Fraction  of  Time  m=>2  ie 
Fnclosed  by  the  Intervals 

0.949 

0.934 

Average  Midpoint  of 
the  intervals 

L  997 

2.284 
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Note  that  Method  #1  appears  to  be  superior  to  the  analysis 
based  on  the  transformed  data.  The  interval  widths  possess  a 
smaller  average  and  much  less  variation. 


Most  experimenters,  using  factorial  experiments  in  trans¬ 
formed  Poisson  data,  employ  an  equal  number  of  observations  per 
factor  level  combination.  This  allocation  is  far  from  optimal 
for  estimating  E(y),  the  response  surface  for  the  untransformed 
data.  Consequently,  it  is  necesaary  to  consider  unequal 
allocation  of  a  sample  to  the  factor  level  combinations  of  a 
complete  factorial  experiment.  We  call  this  type  of  design  a 


This  paper  discusses  the  use  of  least  squares  in  the  analysi 
of  Poisson  data  when  the  data  have  been  collected  from  a  weighted 
factorial  experiment.  Particularly,  we  give  an  expression  for 
y,  the  estimator  of  E(y),  and  give  a  closed  form  expression  for 
its  variance.  This  permits  us  to  choose  optimal  allocation 
of  a  sample  to  the  factor  level  combinations,  or  equivalently. 


to  select  an  optimal  design. 


2.  Fitting  Response  Surfaces  for  Weighted  Factorial  Experiments 
We  will  assume  that  the  expected  mean  Poisson  response, 
E(y),  is  related  to  a  set  of  k  independent  variables,  representi 
the  k  factors  in  a  weighted  factorial  experiment. 


We  assume  that  the  complete  factorial  contains  u  level  combinations 
s 

and  that  E  n  -  n.  The  independent  variables,  x  ,x  , ,,.,x  , 
i»l  *  1  2  r 

are  orthogonal  polynomials  associated  with  the  k  factors  con¬ 
tained  in  the  experiment  and  y^  is  the  average  of  n^  observa¬ 
tions  taken  at  the  ith  factorial  level  combination.  The  form  of 
the  response  surface,  that  is  itB  order,  is  only  limited  by  the 
number  of  levels  included  in  the  weighted  factorial  experiment. 

Let, 

E(Y)  -  X3  (!) 

where 
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Then  it  ie  well  known  that  the  least  squares  fit  to  equation 
(1)  is 


where 


i  -  (X'X>“Vy 


(2) 


The  variance-covariance  matrix  of  Y  is  the  diagonal  matrix, 


E(yi) 

n  9  ial  |  2|  • •  ( ^  8» 

i 

The  estimated  mean  response  at  a  point,  p,  in  the  experimental 
region  of  the  original  k  independent  variables  is  obtained  by 
substituting  the  values  of  these  variables  into  the  orthogonal 
polynomials,  . ,x^,  to  obtain  y  *  ji'J,  where 


Then 


and 


[1 


x  , ,  x  . 
pi  p2 


•  * ,  x  } , 
pr J 


E(y) 


V(y)=  a'  (X,X)"1X,VX(X,X)~1a  . 


(3) 

(4) 
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Letting 


a*‘  -  a' (x'x)-1 

and 

*  “  1.2,. ...a  (5) 

A  A 

we  can  rewrite  y  and  V(y)  as 


*  -  I  Vi  • 

i-1 


V(y) 


l  WiVi 

i-1 


(6) 


Our  objective  ie  to  find  expressions  for  the  weights, 
w^»  i-l,2,...,s,  to  obtain  simple  expressions  for  y  and  V(y) , 

We  can  then  find  the  optimal  allocation,  n,,n  to  the  s 

factor  level  combinations  to  minimize  V(y)  at  a  pre-selected 
point,  p,  in  the  experimental  region.  We  will  give  the 
expressions  for  the  weights  for  first  order  and  complete  factorial 

Jq 

models  for  2  weighted  factorial  experiments  and  for 
second  order  and  complete  factorial  models  for  3  weighted 
factorial  experiments. 


3.  Weights  for  2  Weighted  Factorial  Experiments 

Let  X]/X2' *  *  *  *x)t  rePre8et>t  the  k  linear  orthogonal  polyno- 

k 

mials  corresponding  to  factors  l,2,...,k.  The  complete  2 
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factorial  model  contains  and  terms  corresponding  to  all 
products  of  Xj/X2' ••• ,xk taking  the  variables  one  at  a  time,  two 
at  a  time,,,.,  and  k  at  a  time.  Note  that  this  model  will  con- 

Jf 

tain  exactly  2  terms. 

•  th 

Then  the  vector  associated  with  the  i  factor  level 

Combination,  i-1,2,  ...,s,  is 


I 


C1#eil'®i2' 


eik'eilei2'®ilei3 


where 


®i,k-l®i,k^ 

(7) 


if  factor  j  is  at  the  low  level 
if  factor  j  is  at  the  high  level 
j*>l, 2, . . .  ,k 


Then  it  can  be  shown  that  the  corresponding  weight,  w^, 
i**l, 2 , . . . , s,  xs 


w ,  s  a  u, 
x - i 


«-kn  (i 

2  j=-i 


+  eijxj} 


(8) 


Similarly,  it  can  be  shown  that  w^,  i»l, 2, . . , , s,  for  a  first 


order  model. 


E|y>  ■  so  *  L  8iV 

i=l 


is 


w.»i-  (1  +  e.,x  +  e ,  x  + 
xl  1  i?  2 


-ikV  • 


(9) 
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Thus  for  a  weighted  2x2  factorial,  the  prediction  equations 
for  the  complete  factorial  and  first  order  models  are,  respectively 

y  -  1/4  [(l-x^  (l-x2)y1  -f  (l-x1)<l+x2)y2 

+  (1+x^ (l-x2)y3  +  {l+xx) (l+x2)y4]  (10) 

and 

r  -  l/4[(l-x1-x2)y1  +  (l-x1+x2)y2 

+  (l+x1-x2)y3  +  (l+Xj+XjJy^  (11) 

where  yi  is  the  mean  response  for  n.^  observations  at  the  levels 
of  x^  and  x2  shown  below t 


4.  Weights  for  3  weighted  Factorial  Experiments 

Let  x^,  i-1,2, .. ,,k, represent  the  linear  orthogonal  poly¬ 
nomial  corresponding  to  factor  i  and  let  x^+i  represent  the 
second  order  orthogonal  polynomial  for  the  ith  factor.  Then 

k 

a  complete  3  factorial  model  contains  terms  corresponding  to 
x^,  j=l,2, ...,2k, plus  terms  corresponding  tc  all  combinations 
of  products  of  these  variables  taking  them  two  at  a  time,  three 
at  a  time,.,.,  k  at  a  time,  excluding  terms  that  include  products 


or  k.  Note  that  this  modal  will 


of  xi  and  i-1,2,... 

Ir  2 

contain  exactly  3  terms.  Thus  for  a  complete  3  factorial. 


E(y)  -  0O  +  +  t2x?  +  S3x3  +  p4x4  +  ggX^j 


+  66X1X4  +  P7X2X3  +  P8X3X4 


(12) 


where  x3  is  a  second  order  orthogonal  polynomial  in  x^  and  x^ 
is  a  similar  second  order  orthogonal  polynomial  in  x^. 


Let  e 


ij 


if  factor  j  is  at  the  low  level 
if  factor  j  is  at  the  intermediate  level 
if  factor  j  is  at  the  high  level 
j—l#  2, . . . ,k. 


Then  the  ^  vector  associated  with  the  ith  factor  level  combina¬ 
tion,  1=1, 2, . . . , s,  is 

-i  “  •*"  (3eik_2)  ' 

Qilei2'®iiei3' ***' (3eil“2J  (3ei2”2^  *  ”  *3eik~2*  ^  (13) 

9  k 

(Note  that  contains  3  elements  in  the  same  order  as  they 
appear  in  the  complete  factorial  model.) 

The  second  order  model  for  a  3  weighted  factorial  expert- 

V 

mert  is  obtained  by  deleting  all  terms  from  the  complete  3 
factorial  model  that  are  of  a  third  or  higher  order.  Deleting 
the  corresponding  elements  from  the  vector  for  the  complete 
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factorial  model  gives  the  vector  for  the  second  order  model. 

It  can  be  shown  that  the  weights,  w^,  i-1, 2, . . . , s ,  (s«3  )  for 
the  complete  factorial  model  are. 


w,  -  a*’\j  -  n  h(e  ,x  ) 

1  1  j*i  J 


where 


®J  J  2  2 

h(ei;J,xj)  -  -J1  x_J{lfeijx_j)  +  (1-x^)  (l-ei;J) . 


It  is  difficult  to  obtain  a  simple  expression  for  the 
weights,  w^,  i-l,2,...,s,  for  the  second  order  model  based  on  a 
weighted  3^  factorial  experiment,  but  they  can  be  obtained  from 


the  product. 


wi  " 


2  2 

where  u^  ■  [1,  eil'ejL2»  *  ’  *  ,eiJc' ^3eii”2^  '  * '  ’  *  ^3eik”2^ ' 

*il*i2' *  **#ai,k-iei,k^*  (15) 

For  the  32  weighted  factorial  experiment  we  can  give  the 
prediction  equation  using  the  following  three  functions: 


Q1(X1,X2) 


-J-*1  (1  +  (1  +  x2> 


1  2 
Q2(X1'X2)  “  T  (1  +  X1J  (1  "  X2* 


°3(X1'X2)  "  (1  “  Xl}  “  X2)* 
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Then 


y  -  Q1(-x1,-x2)y1  +  Q2(-x1#>t2)y2  +  Q1(-x1,x2)y3 


+  02<-x2.Xi)y4  +  Q3(x1#x2)y5  +  Q2(x2,x1)y6 


+  Q1(xJL,-x2)y7  +  Q2(x1#x2).y8  +  Q1tx];»x^)y9  ,  (16) 


where  y^,  i*l*2,...,9  is  the  mean  response  of  n^  observations 
taken  at  the  following  levels  of  x^  and  x2< 


i 

XA 

x? 

1 

-l 

-1 

2 

-l 

0 

3 

-l 

1 

4 

0 

-1 

5 

0 

0 

6 

0 

1 

7 

1 

-1 

8 

1 

0 

9 

1 

1 

Similarly  by  defining  the  functions 
lr  2  .  2 


and 


f1(Xi#x2)  -  +  x2  +  x1  +  x2  +  2 

wv  *  iA  -  2x2  +  xi +  !]• 

T3(VV  "K5  -  »t*i  +  X2>] 


-  !] 
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we  can  write  tha  prediction  equation  for  the  second  order  model  of 
2 

the  3  weighted  factorial  experiment  as 

y  -  *1<“x1'"X2)yl  +  'f2(’ICl'X2)y2  +  *l*-*l'x2*y3 
+  y2<“X2#Xl}y4  *  y3<xi»x2^ys  *  y2<X2,Xl^y6 

+  VV’X2)y7  +  VXl'X2)y8  +  fl(Xl#X2)y9  *  (17) 


5.  Sample  Allocation  to  Minimize  V(v)  at  a  Point,  n,  in  the 
Experimental  Region 

We  noted  earlier  that,  for  any  weighted  2*  or  3*  factorial 
experiment, 

*  V  2  E(yi) 

v(y)  "2,  wivi  w^ere  v£  “  " 

i-i  i 


and  w.,  i-l,2,...,B  are  functions  of  x  .,x  _,...,x  the 
*  pi  pz  px 

coordinates  of  a  point  in  the  experimental  region  where  we  want 
to  estimate  E (y) .  We  wish  to  find  n,  ,n.,...,n  ,  the  allocation 

12  S 

of  observations  to  the  factor  level  combinations,  that  minimize 
V(y)  subject  to  the  condition  that  ^l^n^  ■  n.  Assuming  equal  costs 
for  all  observations,  the  minimizing  solution  can  be  shown  to  be. 


and 


n 


E(y£) 

E(ys) 


w 


n 

s 


E  (y  )  w 


(10) 


(19) 
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How  do  you  find  the  value  of  E(y^)*  i«l,2,...,s,  needed  to 
make  an  optimal  allocation  of  the  n  observations  to  the  s  factor 
level  combinations?  Since  we  will  not  know  the  true  response 
surface,  you  can  approximate  these  values  by  making  a  trial  run 
at  the  s  factor  level  combinations.  A  preliminary  computation 

A 

of  y  will  provide  the  approximating  values  for  E (y^) ,  i*l, 2, . . . ,s. 

6.  Comments 

Most  response  surface  explorations  utilize  first  or  second 

order  linear  models.  Factors  are  held  at  two  levels  when  fitting 

a  first  order  model,  and  three  levels  for  a  second  order  model. 

The  weights  and  optimal  sample  allocations  for  these  situations 

were  presented  in  Sections  3,  4,  and  5.  Although  of  lesser 

k  k  k 

importance,  weights  for  a  4  ,  5  ,  or,  in  general  b  ,  b-2,3,4,..., 
can  be  obtained  using  the  procedure  described  in  Section  2. 

A 

The  variance  of  y  for  a  particular  estimation  problem  is 

E(y.)  y, 

obtained  by  approximating  v.  - -  with  —  ,  i=l,2,...,s. 

1  ni  ni 

Note  that  this  approximation  will  be  quite  good  because  y^  is 
the  average  of  n^  observations  at  the  ith  factor  level  combination. 

An  approximate  bound  on  the  error  of  estimation  for  E(y)  is 
then  2/ (y)  where  V(y)  is  the  value  of  V(y)  with  replaced 

by  y^,  i=l,2,...,s.  Finally,  note  that  V(y)  will  vary  depending 
upon  the  location  of  the  point,  p,  where  you  wish  to  estimate 
E(y).  The  experimenter  should  decide  where  maximum  information 
on  E(y)  is  desired  prior  to  making  the  Bample  allocation. 
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1.  INTRODUCTION 

The  basic  problea  considered  la  the  following i  given  the  analy* 
tlcal  fora  of  a  univariate  probability  distribution  function  defined  by 
one  or  more  unknown  parameters  and  given  e  sample  from  the  distribution, 
determine  statistical  inference  Information  on  a  function  of  the  unknown 
parameters.  Of  particular  Interest  are  the  more  difficult  problems  In¬ 
volving  small  samples  for  which  the  accuracy  of  asymptotic  solutlone  is 
doubtful  and  for  quantities  of  interest  which  are  functions  of  more  then 
one  population  parameter.  No  general  solution  exists  to  this  problem  for 
the  classical  frequency  confidence  intervals  but  solution  for  a  Bayesian 
interval  can  often  be  found.  The  main  difficulty  In  the  Bayesian  approac}. 
is,  of  course,  making  a  suitable  choice  of  the  prior  distribution  on  the 
population  parameters  or  quantity  of  interest. 


The  remainder  of  this  article  was  reproduced  photographically  from  the 
manuscript  supplied  by  the  author. 
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The  approach  to  this  problem  discussed  in  this  paper  is  to  employ  a 
Bayesian  technique  which  is,  for  a  particular  class  of  probability  dis¬ 
tributions  to  be  defined  later,  computationally  more  efficient  than  the 
usual  Bayesian  techniques  based  on  the  likelihood  function.  This  then 
facilitates  computer  studies  of  the  exactness  of  Bayesian  confidence 
intervals  from  the  classical  frequency  viewpoint  and  in  atudies  for  the 
possible  development  of  classical  frequency  Intervals  from  the  Bayesian 
distributions. 

The  Bayesian  technique  employed  makes  use  of  the  conditional  proba¬ 
bility  distributions  of  the  estimators  of  population  parameters  given 
the  true  values  of  the  parameters.  This  is  in  contrast  to  the  use  of 
the  likelihood  function  which  gives  the  distribution  of  the  sample  data 
directly.  There  is  consequently,  at  the  outset,  a  general  difference 
in  the  two  Bayesian  confidencing  approaches.  The  confidence  intervals 
derived  from  the  estimator  distribution,  although  exact  in  a  Bayesian 
•ense,  depend  on  the  particular  estimator  used.  Also,  if  the  parameter 
estimators  are  not  sufficient  statistics  then  this  gives  additional 
cause  for  differences  in  comparison  to  the  likelihood  approach.  In  the 
work  to  be  discussed  in  this  paper  only  distributions  of  the  maximum 
likelihood  estimators  were  studied  because  of  the  desirable  properties 
of  these  estimators. 

Two  examples  of  the  n®n«al  problems  of  interest  are  given  as  follows: 

a.  For  the  Weibull  distribution  f(x;a,S)  is  given  as 


3-1  -Cx/a) 

f(xja,8)  •  (&)  (*)  • 

a  a 


6 


(1) 
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The  mean  u  is  then  given  by 


M  a  nl*  (1  +  1/8)  a  g  (a, 8)  (2) 

where  confidence  intervals  are  required  for  the  mean. 

t>.  For  arbitrary  first  failure  distribution  F(tia,8)  the  "ideal 
repair"  interval  reliability  at  arbitrary  tine  t  for  interval  t  is 
given  by  the  expression 

R(t(r;a,8)  ■  1  -  P(t  ♦Tja,8) 


♦  £*[1  -  F(t  x;a,B) ]h(x;a,S)dx 
where  h(x;a,S)  is  determined  from  the  renewal  equation 


(3) 


h(xjo,8)  a  f(x;o,8)  ♦  /Xf(x  -  yja,8)  h(y;o,3)dy 

b  (4) 

As  can  be  seen  the  interval  reliability  for  given  t  and  t  is  a  function 

of  the  unknown  distribution  parameters  a  and  0.  Confidence  intervals 

aro  then  required  for  the  interval  reliability. 


2.  ONE  PARAMETER  DISTRIBUTION 

The  case  of  the  one  parameter  distribution  is  relatively  straightforward 
and  generally  offers  no  difficulty.  It  is  used  here  to  introduce  some  of  the 
basic  ideas  in  the  construction  of  confidence  intervals.  For  a  given  sample 
of  size  n  from  the  distribution  F(x;a)  various  techniques  can  be  used  to 
determine  an  estimate  a  of  the  parameter  a.  Maximum  likelihood  is  basically 
the  estimating  technique  used  in  the  work  described  in  this  paper.  It  is 
also  a  technique  that  often  leads  to  minimum  variance  for  estimates  and  has 
other  desirable  properties  (Reference  1,  Section  33,2), 
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One  approach  to  determining  classical  frequency  confidence  intervals 
for  either  a  or  a  suitably  behaved  function  g(a)  is  to  generate  the  con¬ 
ditional  probability  distribution  of  a  (or  g(a))  as  a  function  of  a 
(or  g(a))s  that  is  P«  (a;o»)  for  the  estimator  a.  One  then  finds  limits 
(a)  and  y2  (&)  for  a  as  a  function  of  a  such  that  the  probability 
that  a  falls  between  Yj  and  y2  ia  fixed  at  the  desired  percent  confidence 
level  p.  The  limits  y^  (a)  and  y2  (a)  form  curves  on  the  (a, a)  plane  with 
u  being  the  independent  variable.  One  can  now  go  through  various  arguments 

*  at 

to  show  that  for  a  given  estimator  a,  the  values  of  a  for  which  a"Yj(ai) 
and  a>Y2(a2)  *Te  th®  upper  and  lower  confidence  limits  on  a  (Reference  1, 
Section  34.2).  With  confidence  intervals  constructed  in  this  manner, 
independent  trials,  where  each  trial  consists  of  drawing  say  a  sample  n 
from  the  given  population,  would  yield  confidence  limits  that  covered  the 
true  value  an  average  of  p  percent  of  the  trials.  In  this  case  the  true 
parameter  can  be  regarded  as  fixed  or  allowed  to  vary  in  an  arbitrary 
fashion. 

In  the  Bayesian  approach  the  parameter  a  is  considered  itself  to  be  a 
random  variable  having  a  certain  a  priori  distribution  fj(a).  One  then 
uses  Bayes'  theorem  to  determine  the  a  posteriori  distribution  f2(a)  of  a 
for  a  given  sample  outcome  making  use  of  the  conditional  distribution  of 
the  sample  as  a  function  of  a.  The  conditional  distribution  of  the  sample 
for  given  a  is  often  assumed  to  be  the  likelihood  function  for  the  more 
complicated  problems.  The  conditional  distribution  of  an  estimator  a  for 
given  a  can  also  be  used  to  generate  the  a  posteriori  distribution  of  a. 
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Bayes'  theorem  for  both  of  those  esses  are  given  as  follows: 


f2(o /ID  ■  Cx  fx(a)  L  CTja) 


(5) 


and 

t2  (a/a)  m  c,  f.  (a)  f  (aja) 

221  (6) 

« 

in  which  L  is  the  likelihood  function,  IT  is  the  sample  outcome,  a  is  the 
estimator  of  a  and  and  C2  are  normalising  constants,  confidence 
intervals  can  then  be  constructed  directly  from  the  distribution  f2  (a/x) 
or  f2  (a/a). 

The  main  difficulty  in  the  Bayesian  approach  is  to  determine  the 
a  priori  distribution  fj  (a),  particularly  when  there  is  very  little 
and/or  questionable  prior  data  on  the  true  parameter  values  one  could 
possibly  expect. 

The  likelihood  function  is  most  often  used  in  constructing  Bayesian 
confidence  limits  because  there  is  no  necessity  for  deciding  what 
estimator  to  use  and  because  of  the  general  ease  in  theoretically 
constructing  a  solution  for  confidence  limits.  Also,  determining  the 
two-dimensional  function  f  (a;a)  can  be  tedious  if  analytical  solutions 
are  not  possible.  As  will  be  shown  shortly,  however,  the  quantities  a 
and  a  can  often  be  transformed  into  a  single  parameter  ag,  the  probability 
distribution  of  which  is  independent  of  the  true  parameter  a  [eg,  ag  *  a/a 
for  the  exponential  distribution).  The  problem  is  thus  reduced  to  the 
determination  of  a  one-dimensional  function  f  (ag)  for  given  sample  sizo  n. 


This  type  of  transformation  becomes  much  more  important  in  the  multi* 
parameter  problem.  Also,  note  that  the  distribution  of  yields 
directly  the  classical  frequency  interval  for  o  which,  in  this  instance, 
is  equal  to  the  Bayesian  interval  determined  from  f2  (ajo)  with  uniform 
prior  on  a. 

'Che  primary  reason  for  approaching  the  Bayesian  confidencing  problem 
using  the  estimator  distribution  is  that  once  the  distribution  of  the 
estimator  is  available  for  given  sample  else  n,  the  computer  time  for 
constructing  confidence  intervals  was  foind  to  be  much  shorter  than  when 
using  the  likelihood  approach  for  the  more  complicated  problems  of 
interest.  This  then  facilitated  computer  studies  of  the  exactness,  from 
a  classical  frequency  viewpoint,  of  the  Bayesian  intervals  and  in  studies 
of  procedures  for  generating  classical  frequency  intervals  from  the 
Bayesian  probability  distributions, 

3.  TWO  OR  MORE  PARAMETER  DISTRIBUTIONS 

Only  the  two  parameter  problem  will  be  considered  in  what  follows  to 
simplify  the  discussion  although  extension  to  more  than  two  parameters  is 
straightforward.  In  many  instances,  however,  only  the  two  parameter 
problem  is  practical  to  so've. 

When  confidence  intervals  are  desired  for  a  function  g  of  two  unknown 
population  parameters,  the  classical  frequency  approach  discussed  in  the 
previous  section  cannot  be  applied  in  general.  For  example,  the  distribu¬ 
tion  of  the  estimator  of  say  g  -  g(a,B)  does  not  generally  depend  directly 
on  the  true  value  of  g  but  rather  on  both  of  the  unknown  parameters  a  and  s. 
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Hence,  for  a  fixed  true  value  of  g,  a  end  0  can  take  on  different  valuer 
yielding  different  distribution*  of  g,  There  are  exception*  to  this  auch 
a*  the  first  failure  reliability  for  the  two  parameter  Weibull  distribution 
(Reference  4). 

In  the  Bayesian  approach  the  key  to  solving  the  two  parameter  problem 
is  the  determination  of  the  bivariate  distribution  f  («,!;<*,  8)  for  the 
estimators  a  and  0.  Once  the  bivariate  distribution  is  available  for  a  end 

A 

0  as  a  function  of  a  end  S  and  by  assuming  soma  prior  distribution  on  the 
unknown  parameters  the  distribution  can  be  transformed  as  in  the  one  para¬ 
meter  case  into  a  posteriori  distribution  of  o  and  B  for  given  a  end  8. 

The  distribution  of  a  function  g(a,8)  of  the  parameters  a  and  8  can  then 
be  determined  from  f(a,8ja,B)  (Reference  2,  Chapters  S  and  7).  The  deter¬ 
mination  of  exact  Bayesian  confidence  intervals  can  then  be  obtained  as  in 
the  one  parameter  case. 

The  function  f(a,8ja,B)  is  a  four-dimensional  function.  In  the  next 
section  the  transformation  of  the  four-dimensional  into  a  two-dimensional 
problem  is  considered.  This  then  greatly  simplifies  the  computational 
aspects  of  this  overall  problem. 

4.  A  GENERAL  TRANSFORMATION  ON  ESTIMATORS: 

Consider  the  maximum  likelihood  estimators  of  the  parameters  a  and  8  for 

a  given  sample  xi#  i  ■  1,  2,...,  n  taken  from  a  population  with  distribution 

f(x;a,8).  The  likelihood  fwetion  L  in  this  case  is  given  by 
n 

L  •  T P  f  (Xija.B) 

i-l  (7) 
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where  the  estimator*  a  and  0  aaximiie  the  function  L.  Hie  solution  for 
ci  end  0  in  this  cess  depend  only  on  the  sample  values  xj  and  the  functional 
form  f(x^{a,S)  or  alternately  on  the  functional  form  of  the  emulative 
distribution  function  P(x^;a,l).  Estimators  other  than  maximum  likelihood, 
of  course,  could  fit  this  criteria. 

Consider  next  the  original  cumulative  probability  distribution  P(x;a,B) 
for  some  given  true  a  and  0  and  from  which  the  sample  is  generated.  It 
can  bo  shown  that  for  an  arbitrary  sample  value  x^  the  functional  value  of 
R(*iJ*#8)  evaluated  at  the  point  x^  is  itself  a  random  variable,  call  it 
which  is  distributed  uniformly  on  the  interval  (0.  1)  Reference  3.  Pai*  313). 


In  this  case  then  Rj  •  Ffx^ja.B)  which  is  independent  of  a  end  0.  Using  this 
expression  to  solve  for  x^  gives 
x.  ■  F^CR.ja.S). 

1  1  («) 

Substituting  this  equation  into  the  function  F(x£;a.B)  used  to  determine  the 

estimators  n  and  8  finally  gives 

FCx.ja.B)  ■  FCF"1(Ri;«,0);5,8) 

1  (9) 

If  this  function  can  be  transformed  into  the  function  F(^(R^);a(.08)  which 

has  the  same  analytical  form  as  P(x^;a.S).  then  it  is  clear  that  the 

estimators  as  and  S8  (or  any  function  of  them)  in  this  expression  are 


statistically  independent  of  a  and  3  since  the  Z(Rj)  are  independent  of 
a  and  S.  In  general  a,  ■  u(a, 0,5,0)  and  05  -  v(a, 0,5,0),  This  means 
that  it  is  only  necessary  to  generate  the  joint  distribution  of  5#  and  8, 
to  completely  define  the  conditional  frequency  distribution  f(5,0;a,0). 
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I 


k 

tj 

l 


Thus  the  four-dimensional  problem  is  reduced  to  s  two-dimensional  problem. 
The  key  to  this  problem  is  transforming  F(P"1  (R^;a,B) ;a,B)  into  the  form 
PCZCRiJsSj.S,). 

5.  EXAMPLES  OP  THE  GENERAL  TRANSFORMATION 


a.  Uniform  Distribution. 

The  cumulative  distribution  ftmction  for  this  case  is  given  by 


0  j  x<a 

F(x;a,S)  “  { (x-a)/(B-a);  a~x<8 
1  i  »B 


Letting  ■  F(xjja,8)  and  solving  for  Xj  gives 


(10) 


x,  >«  R.  (8-a)  +  a 

1  (11) 
over  the  interval  of  interest 

Substituting  into  the  expression  F(xi;s,8)  yields  the  fvmction 

F(Zi(Ri);isfBs)  -  (Z^o,)/ «>,-•,) 

where  Zi(R^)  ■  Rjj 

os  ■  (a-a)/CB-a); 

\  *  (9-a)/C8-a), 


It  is  only  necessary  then  to  generate  the  joint  distribution  of  a  and  0 

*  s 

to  completely  define  the  distribution  of  a  end  3  fur  given  a  and  6. 
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b.  Guebel  Extremal  Distribution  of  Type  I, 

In  this  csss 

P(x)  ■  1  -  exp  {-  exp  [(X-a)/S]) 

Using  ths  previously  described  procedures  it  esn  be  shown  that 
ZCRj)  -  ln[-  In  (1  -  Rt)l  $ 


s,  «  (5/s). 


c,  Weibull  Distribution. 

For  the  two  paraawter  Weibull  distribution 

P(x)  •  1  -  exp  (-<£)*} 
a 

fron  which  is  obtained 

Z^)  -  -ln(l-R^) ; 

A 

a,  -  (a/a)  ; 


(IS) 


(14) 


(15) 


5,  •  (5/s) • 


(16) 


The  results  given  by  the  above  expression  was  derived  by  Thoman,  et  al 
(Reference  4)  in  their  work  on  inferences  for  the  parameters  of  the 
Weibull  distribution. 


6.  INFERENCES  OK  FUNCTIONS  OF  TUB  WEIBULL  PARAMETERS 

The  two  parameter  Woibull  distribution  proves  an  interesting  example 
for  application  of  the  statistical  theory  just  presented.  First,  general 
analytical  solutions  for  the  distribution  of  the  maximum  likelihood 
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estimators  are  not  available.  The  bivariate  diatribution  of  a#  and  Sg 
in  this  instance  must  therefore  be  determined  numerically.  This  was 
done  using  Monti  Carlo  simulation  for  different  sample  sizes.  Specifi¬ 
cally  the  frequency  distribution  f(x,y)  where  x  ■  (a/o)^  and  y  *  S/B 
was  generated  for  sample  sizes  n  -  5,  8  and  20  using  20(000,  20,000 
and  10,000  points  respectively.  Other  sample  sizes  could  of  course 
have  been  considered.  Once  the  distribution  f(x,y)  is  generated  for 
a  given  sample  size,  Bayesian  confidence  intervals  on  functions  of 
the  MpJAmj  parameters  can  be  computed. 

A  particular  function  considered  for  application  was  the  mean 
u  -  «r(l  ♦  l/B).  For  given  maximun  likelihood  estimates  of  the 
parameters,  a  and  S,  and  assuming  a  uniform  prior  on  a  and  B,  the 
probability  distribution  of  the  mean  from  which  confidence  intervals 
can  be  computed  is  given  as  follows: 

Fy(Z)  -  Prob  [n<Z] 

m  m 

•  /  /  yjdxdy 

b  y 

where  ■  [  ftfll+y/S)  ,.]^t 

Z  (17) 

For  example,  an  80%  Bayesian  confidence  interval  can  be  determined  by 

solving  for  Z1  and  Zu  such  that  P^Zj)  -0.1  and  Py(Zu)  -  0.9,  In  this 

case  Zj  and  Zu  are  the  lower  and  upper  confidence  limits  respectively. 
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7,  RESULTS  OP  COMPUTATIONS 


A  number  of  computations  war*  performed  using  the  above  described 
procedures  for  confidencing  the  Wiibull  mean.  Some  of  the  results  of 
these  computations  are  described  as  follows: 
a.  Exactness  of  Confidence  Intervals. 

Monte  Carlo  simulation  was  used  to  generate  sets  of  5000  and 
1000  samples  of  sizes  5  and  8  from  a  Weibull  distribution  with  fixed 
true  mean  u  equal  to  1.0  and  a  uniform  prior  on  the  shape  parameter  8. 

Por  each  plven  sample,  the  function  Fy(Z)  in  equation  (17)  was  evaluated 
for  Z  *  1.0.  If  the  Bayesian  confidence  intervals  determined  using  PM(Z) 
aro  exact  then  the  computation  of  this  function  at  Z  ■  1  for  the  given 
samples  should  yield  a  uniform  distribution  on  the  interval  [0,  1].  This 
interval  was  divided  into  ten  parts  and  a  chi  square  test  was  performed 
on  the  resulting  data.  The  hypothesis  that  the  distribution  of^(z  •  1) 
was  uniform  could  not  be  rejected  down  to  the  50%  confidence  level  for  the 
cases  considered. 

The  Bayesian  intervals  wore  also  studied  to  determine  exactness  from  a 
classical  frequency  viewpoint.  In  this  instance  the  shape  parameter  ft  was 
held  fixed  as  well  as  the  mean  u.  A  number  of  Monte  Carlo  simulations  were 
performed  with  6  fixed  at  various  values.  For  values  of  8  greater  than 
about  7.0  the  Bayesian  intervals  were  very  nearly  exact.  For  smallex  values 
of  8,  particularly  near  1,0,  however,  the  intervals  deviated ftaMn  being 
exact  although  not  by  a  great  degree. 


A  number  of  approaches  wart  takan  to  attempt  to  transform  tha 
Bayesian  intervals  Into  exact  classical  frequency  intervals  for  all 
valuaa  of  B.  Tha  moat  successful  approach  studied  thua  far  involvaa 
tha  uae  of  a  biailng  factor  c  on  tha  true  naan  which  la  a  function 
of  tha  unbiaaad  naxinun  likelihood  eatiaator  for  8,  In  tha  Monte 
Carlo  triala  for  axactnaaa  deacribed  abova,  inataad  of  evaluating 
Fy(Z  ■  ut)  where  ut,  the  true  naan,  is  fixed  at  1.0,  tha  function 
Fp(Z  *  utC)  ia  evaluated  and  axaetnaaa  is  than  cheeked  for  this 
function.  For  tha  Weibull  mean,  tha  function 

C  ■  1.0  ♦  a  ♦  b  , 

(I  (f  )Z  (II 

in  which  6  ia  tha  unbiased  estimator  for  6,  gave  confidence  intervals 
that  were  very  nearly  exact  for  fixed  vpiues  of  8  ranging  from  1.0  to 
10.0,  For  a  sample  alee  of  8,  for  example,  the  empirically  derived 
values  of  a  and  b  required  in  equation  (18)  were  approximately  0.055 
and  0.015  respectively.  In  this  case  values  of  c  equal  to  1.07,  2.0116 
and  1.0056  are  obtained  for  8  equal  to  1.0,  5.0  and  10.0  respectively. 
The  Bayesian  confidence  limits  when  divided  by  C  yield  confidence  limits 
which  are  nearly  exact  from  a  classical  frequency  viewpoint.  No  general 
conclusion  will  be  made  at  this  time  on  this  overall  approach  since  work 
on  this  problem  is  not  complete.  Further  studies  are  to  be  conducted  in 
the  future. 


b,  Comparison  of  Confident  Limit*. 

Bayedsn  confidence  intervals  using  uniform  priors  were  deter¬ 
mined  on  Monti  Carlo  sample  data  using  both  the  likelihood  fwiction 
and  the  estimator  distribution .  The  samples  were  generated  assuming 
true  parameter  values  \t  ■  1  and  S  ■  1  and  3.  Tables  1  and  2  present 
some  of  the  results  of  these  computations  for  a  sample  sise  of  5. 

No  major  conclusions  can  be  made  based  on  the  data  generated 
thus  far.  It  does  appear,  however,  that  the  two  Bayesian  confidence 
limits  are  similar  in  their  general  behavior  although  some  differences 
are  observed  in  their  values  depending  on  the  value  of  B  as  can  be  seen 
from  the  data  in  Tables  1  and  2, 

Another  general  observation  is  that  the  computer  times  required 
to  generate  the  Bayesian  Intervals  based  on  estimator  distribution, 
once  the  bivariate  distribution  of  parameter  estimators  is  generated, 
were  considerably  loss  than  for  the  Bayesian  intervals  based  on  the 
likelihood  function.  An  order  of  magnitude  difference  was  observed  for 
many  of  the  confidencing  problems  considered. 

Table  3  lists  some  results  comparing  the  Bayesian  and  classical 
frequency  lower  90%  confidence  limit  for  a  sample  sise  of  8.  As 
indicated  previously  the  classical  frequency  intervals  for  the  values 
of  B  considered  agree  more  closely  for  the  larger  values  of  0. 

c.  Moments  of  the  Maximum  Likelihood  Main. 

It  is  of  interest  to  observe  the  asymptotic  behavior  of  estimators 
as  a  function  of  sample  size.  It  is  known  for  example  that  maximum  likeli¬ 
hood  estimators  are  asymptotically  normal.  Table  4  lists  the  resulting 
computation  of  the  variance,  skewness  and  kurtosis  of  the  mean  as  a  function 


of  0  and  san^le  sir*.  On*  proparty  of  tha  normal  distribution  is  that 
it  has  a  skewness  of  taro  and  kurtosis  equal  to  3*0,  As  can  ba  saan 
for  8>1  tha  maximise  likelihood  estimators  for  tha  naan  hava  approached 
tha  normal  distribution  avan  for  tha  sampla  sita  of  5. 
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TAB  LB  1:  TWO-SIDED  BAYESIAN  CONFIDENCE  INTERVALS  ON  THE 

WBIBULL  MEAN  FOR  6  -  1.0:  CONFIDENCE  LEVBL  -  80% 
W  -  1.0  AND  SAMPLE  SIZE  -  5. 


CONDITIONAL  DISTRIBUTION  USED 


SAMPLE 

NUMBER 

MAXIMUM  LIKELIHOOD  ESTIMATOR 
DISTRIBUTION 

LIKELIHOOD  FI  NOTION 

LOWER 

UPPER 

LOWER 

UPPER 

1 

.S77 

1.103 

.638 

1.062 

t 

2 

.623 

1.502 

.699 

1.399 

3 

.320 

2.561 

.414 

2.430 

4 

.635 

1.136 

.687 

1.097 

5 

.721 

3.442 

.874 

3.059 

6 

.234 

,743 

.271 

.674 

7 

.886 

4.580 

1.083 

3.888 

8 

.402 

1.202 

.463 

1.078 

9 

.379 

1.712 

.455 

1.461 

10 

.502 

1.285 

.565 

1.125 
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TABLE  2 1  TWO-SIDED  BAYESIAN  CONFIDENCE  INTERVALS  ON  THE 

WE I BULL  MEAN  TOR  6  •  3.0:  CONFIDENCE  LEVEL  >  80%, 
u  -  1.9  AND  SAMPLE  SIZE  -  5. 


CONDITIONAL  DISTRIBUTION  USED 


SAMPLE 

NUMBBR 


MAXIMUM  LIKELIHOOD  ESTIMATOR 
DISTRIBUTION 


r  «— rifcaban.. 


TABLE  3t  BAYESIAN  AND  CLASSICAL  LOWBR  DO*  CONFIDENCE  LIMITS 

ON  THE  WE I BULL  MEAN  FOR  SAMPLE  SIZE  OF  8  AND  TRUE  j 

■i 

MEAN  OP  I.O. 

. . . . . i 


SAMPLE 

NUMBER 

TRUE  S 

-  1.0 

TRIIU  8 

■  3.0 

TRUE  6 

-  5.0 

BAYESIAN 

CLASSICAL 

BAYESIAN 

CLASSICAL 

BAYESIAN 

CLASSICAL 

1 

.562 

.503 

.737 

.715 

.809 

.795 

2 

.785 

.735 

.910 

.893 

.939 

.928 

3 

.902 

.806 

.864 

.838 

.891 

.875 

4 

.922 

.858 

.947 

.927 

.959 

,947 

5 

.562 

.538 

.858 

.847 

.916 

.909 

6 

1.243 

1.136 

1,005 

,980 

.985 

.970 

7 

.538 

.516 

.848 

.837 

.910 

.903 

8 

.841 

.767 

,879 

.856 

.908 

.894 

9 

.686 

.612 

.782 

,758 

.837 

.822 

10 

.639 

.594 

.836 

.818 

.889 

.878 
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MOMENTS  OF  HE  MAXIMUM  LIKELIHOOD  JEW* 


DYNOSS  -  DYNAMICALLY  OPTIMIZED  SMOOTHING  SPAN 


Roberto  Fierro 

Analysis  and  Computation  Division 
White  Sands  Missile  Range,  New  Mexico 


FOREWORD. 


The  work  covered  by  this  report  was  done  In  the  Software  Section, 

Data  Reduction  Branch,  Analysis  end  Computation  Division,  White  Sends 
Missile  Range.  This  study  la  part  of  a  continuing  program  to  determine 
optimum  methods  for  the  reduction  of  missile  data  at  White  Sands  Missile 
Range.  The  general  subject  of  filtering/smoothing  is  very  old  in  the 
field  of  data  reduction,  though  the  bulk  of  the  work  reported  here  was 
accomplished  during  the  year  1970. 

ABSTRACT. 

This  report  describes  a  dynamic  recursive  least  squares  polynomial 
smoothing  span  technique.  Details  of  theory,  design,  and  operation  of 
the  technique  are  given.  This  method  is  compared  with  classical  least 
squares  fixed  span  smoothing.  The  technique  is  not  limited  to  any  given 
type  of  data  but  is  amenable  to  time  series  measurements  from  any  given 
source.  Typical  measurements  come  from  radar,  fixed  camera,  or 
tracking  camera.  The  computing  time  is  extremely  small  and  Invariant  with 
length  of  span.  DYNOoS  la  being  applied  to  missile  trajectory  data  for 
some  current  testing  programs  at  White  Sands  Missile  Range.  By  every 
Important  criterion,  DYNOSS  is  always  better  than  the  least  squares 
fixed  span  smoothing  method. 


The  remainder  of  this  article  was  reproduced  photographically  from  the 
authors  manuscript. 


Z.  INTRODUCTION. 

The  problem*  of  anoothlng/flltarlng  data  ara  old  in  tha  fiald 
of  data  induction.  For  a  good  nuabAr  of  year*,  tha  fix ad  apan  laaat 
aquarea  smoothing  nsthod  haa  baan  aatiafaetory  and  no  significantly 
new  methods  or  approaehaa  hava  baan  found  nacaaaary.  However,  tha 
development  of  axtranaly  high  aeealaratad  nultl-stage  alaallaa  makes 
nacaaaary  a  re-examination  of  paat  anoothlng/flltarlng  tachnlquaa. 

Tha  purpoaa  of  tha  praaant  papar  la  to  show  that  tha  dynamic  apan 
rtoo  thing  approach  off ara  aignif leant  advantage*  over  fixed  point 
■soothing  in  tha  generation  of  nisslla  state  aatinataa.  Fur tha more, 
tha  thaory  introduced  la  applicable  to  a  wide  variety  of  estimation 
problena.  With  tha  advantagaa  that  DYNOSS  off ara,  it  cannot  te  denied 
recognition  aa  a  aignif leant  advancement  in  tha  atata  of  tha  art. 

In  thla  report,  wa  refer  to  flltera  and  filtering;  however,  tha 
point  being  evaluated  la  tha  saoothad  point.  The  filtered  data  (real 
tine  point)  can  be  obtained  by  propagating  the  aaoothad  raeulta  to  tha 
and  point . 

To  coapar*  tha  performance  of  flxad  apan  filtara  and  DYNOSS, 

error  free  trajectory  data  waa  filtered  over  a  fixed  apan  and  than 

through  DYNOSS.  Tha  traj actor laa  uaad  to  exerdae  the  flltera  ware 

a  Standard  Athena  trajectory  and  an  axtraaialy  high  accelerated  slau- 

latad  trajectory.  Tha  Athena  data  conalata  of  a  flrat  stage  burnout, 

aacond  stage  ignition  and  burnout  and  tha  beginning  of  a  ballistic 

2 

trajectory.  Tha  axtranaly  high  aeealaratad  (up  to  10,000  ft/aac  ) 
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launch  consist*  of  short  ignition  periods,  and  abrupt  changoa  in 
acceleration.  The  standard  position,  velocity  and  acceleration 
profile  of  the  Z  component  of  the  trajectories  are  shown  in  Fig*. 
1,2,  3,  14,  15,  and  16.  The  Athena  data  vaa  generated  end  used 
at  20  sanples  per  second  while  the  High  Acceleration  data  was  at 
SO  sanples  per  second.  Since,  recovery  of  the  true  signal  ie  also 
a  function  of  noise  content,  randon  error  was  then  added  to  the 
standard  position  data  and  ths  filtering  repeated. 

The  results  of  experinents  to  conpare  the  DTN0S8  technique 
to  that  of  fixed  least  aquares  smoothing  are  described  in  detail 
with  graphs. 

The  author  wishes  to  express  his  appreciation  to  Darold  Comstock 
for  his  outstanding  contributions  to  this  study. 


the  developnent  of  DYH088  (acronyn  for  Dynanlcally  Optlnlsed 


Snoothing  Spans) 


••  a  result  of  studying  the  spsaiflo  control 


conditions  that  could  be  applied  to  tha  aquations  for  tha  weighted 
least  squares  recursive  astlnation  (WLS&B)  filter. 

The  HLIRX  fllter/snoothar  can  be  described  by  the  equations! 

X  -  X  +  (0  +  Bfc  WH)'1  H*W  (Z-HX), 

2  t  -1 

0  -  (0  +  h'wh)  , 

Kn  "  *xf 


2  t  -1 

-  (to.  4  >  , 


where: 


X  ■  optlalsed  state  vector, 

A 

X  ■  predicted  state  vector, 

G  ■  Weight  natrix  for  X,. 

W  -  Weight  natrix  for  Z, 

Z  -  observations, 

2 

a  m  variance-covariance  natrix  for  X, 

4  ■  transition  natrix, 

and  H  is  the  observation  natrix  which  relates  the  observations  to 

the  state  estinates. 

Thee  we  see  that  a  previous  satinets  nay  be  updated  by  the 
seount  that  a  new  observation  contributes. 
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The  major  problaa  araaa  in  ualng  tha  WL3U  aquationa  ara  la 
determining  U  and  In  lmpoalng  ehaeka  and  eoutrola  whan  tha  tranaltlon 
■atria  doaa  not  adequately  daaerlba  tha  prooaae. 

In  tha  early  atagaa  of  our  atuay  tha  eoaparlaon  of  elaaaleal 
laaat  aquaraa  to  daak  calculated  racuralva  aatlaatlona  providad  ua 
with  furthar  insight  Into  tha  WLSRE  f 11 tar  and  aldad  la  opacifying 
tha  control  condltiona  and  aquationa  to  ba  uaad.  It  waa  at  tkla 
point  that  wa  raaliaad  thara  axiatad  a  dlra  naad  of  a  tachnlqua  for 
determining  tha  optimum  filtering  a pan. 

Data  to  ba  amoothad  by  polynomial  flltara  ara  aanplad  at  a  high 
anough  rata  to  conform  to  tha  flltar  aa  long  aa  tha  numbar  of  point 
amoothlng  (K)  la  not  too  largo.  Ragardloaa  of  tha  numbar  of  obaar- 
vat Iona,  tha  optlmlnad  flltarad  data  will  ba  obtained  uaing  tha 
largaat  aat  (apan)  of  input  data  that  conforma  to  tha  filter  daaign. 
Tharafora  it  la  highly  daelrabla  for  aach  point  to  find  tha  largaat 
aat  of  data  which  includae  that  point  and  conforma  to  tha  filter 
a pacification. 

If  tha  racuralva  aquation  la  uaad  aa  atatad.  than  tha  apan  will 
ba  from  tha  flrat  data  point  through  tha  lataat  data  point.  Thu*  it 
bacomaa  nacaaaary  to  find  a one  method  of  raatricting  the  apan.  Of 
tha  poaaible  waya  of  raatricting  K  (Numbar  of  data  polnta  uaad  In 
tha  flltar),  thara  ara  thraa  which  wa  believe  ahould  ba  mentioned. 
Theee  thraa  are  thruatlng  error a,  fixed  K  and  DYN088, 

Tha  true  modal  ueually  conalata  of  many  more  variable*  than 
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those  listed  la  X.  Thar afore,  tha  predicted  etete  will  be  slightly 
In  error,  since  these  variables  were  not  considered  when  X  at  t  waa 

A 

transformed  to  X  at  t  ♦  1.  All  thase  variables  usually  cannot  ba 
extracted  free  tha  data  nor  obtained  froa  other  sources  with  the 
desired  deiree  of  accuracy.  However,  one  way  In  which  they  can  ba 
taken  Into  consideration  la  to  Include  a  throating  error  in  0,  the 
weight  matrix  of  the  predicted  state.  This  affectively  reducee  the 
filtering  span  by  assigning  a  smaller  weight  to  the  predicted  state, 
or  It  nay  be  viewed  as  a  relatively  larger  weight  being  assigned  to 
the  observation.  In  using  throating  errors,  It  Is  usually  necessary 
to  obtain  the  errors  by  nonltorlng  accelerations,  altitude,  or  sons 
ocher  alsslle  paraaeter.  Tha  main  pitfall  with  this  approach  Is 
that  we  really  do  not  Include  tha  thrusting  force  per  se  In  the 
equation  of  state  used  to  generate  the  state  eatlaates.  Although 
the  augmentation  of  tha  C  matrix  with  a  throating  error  does  enhance 
the  value  of  tha  state  estimates.  Another  drawback  Is  that  usually 
che  cheeka  made  are  for  a  given  standard  missile  system.  However, 
wa  do  not  know  what  checks  to  provide  for  tha  missile  systnas  of 
tha  future  or  the  missile  that  does  not  conform  to  the  standard. 

The  weight  matrix  of  tha  predicted  state,  C,  may  be  varied 
such  that  one  Affectively  uses  a  fixed  K.  The  main  drawback  with 
using  a  fixed  K  Is  that  one  will  obtain  a  misfit  of  the  data  during 
such  events  as  Ignition  and  burnout.  The  Ideal  situation  would  be 
to  vary  the  weight  matrix  0  such  that  K  will  be  dynamically  adjusted 
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to  yield  optima  results.  This  may  also  be  viewed  at  aaloetlat  tha 
optima  data  opaa  or  tot  alia. 

OVSMI  does  Ml  axhibit  tha  veaknasssi  of  tha  otbaz  two  aatboda. 

In  faat  0YM08I  oaa  detect  all  avanta  with  raaarkabla  accuracy  at^ 
optialaa  tha  data  through  a  dynaalo  proeaaa  which  aalaeta  tha  ap- 
propriata  K.  Fur  the  more,  DTK0I8  will  handle  any  present  and  future 
nisslle  ayataa  without  any  mdifieationa  and/or  huaan  intervention. 

Xn  utilising  the  WLIRX  aquations)  wa  need  a  procedure  to  cow 
putOi  W,  tha  weight  matrix  for  Z.  Tha  possible  ways  to  coepute  W 
are  the  following I 

(1)  Coepute  W  based  on  ehe  difference  between  the  observation 

and  a  predicted  value  of  tha  observation. 

(2)  Comute  W  fron  W  -  and  6  -  f  (K,  o^). 

°Z 

Coeputlng  W  by  the  first  aethod  has  certain  disadvantages.  Since 
V  la  based  on  the  difference  between  predicted  value  of  the  observation 
and  tha  observation!  tha  error  in  tha  weight  baconeo  a  function  of 
tha  error  in  both  values.  Thus  an  error  In  tha  predicted  value  re¬ 
sults  in  an  erroneous  weight.  This  bacons a  critical,  especially  near 
event  tines,  whan  tha  predicted  value  cannot  reflect  tha  event  and 
as  a  result  the  latest  observation  is  Assigned  a  falsa  weight.  If 
this  process  Is  continued  over  several  points,  then  the  event  nay  be 
ranoved  fron  the  filtered  date. 


The  second  approach  has  Che  disadvantage  that  all  observations 

are  given  the  aaae  weight.  Further  Investigation  reveals  that  this 

is  not  very  such  of  a  disadvantage,  since  historically  it  has  .bean 

observed  that  the  f liter* i  output  of  a  set  of  deta  with  randoie  errors 

is  not  significantly  ir. voided  if  the  weights  are  chosen  as  being 

equal,  such  as  in  *v<v.<rging  a  set  of  nuabers.  The  key  point  being 

that  the  errors  are  random.  This  is  the  case  for  the  types  of  data 

2 

being  considered.  This  approach  has  the  advantage  that  a ^  can  be 
factored  from  G  and  W,  leaving  (G  +  H*  WH)'  as  a  function  of  K  and  At. 

s  •  j 

Thus  the  (G  +  H  WH)  matrix  doos  not  have  to  be  Inverted  every  tine 
but  can  be  predicted  based  on  the  known  values  K  and  At. 

B.  Introduction 

The  purpose  of  DYN088  is  to  optimise  data  through  a  dynamic 

process  which  will  Increase  or  decrease  the  span  sine  depending  on 

the  outcona  of  a  99X  confidence  level  7-test.  Another  extremely 

attractive  feature  is  the  use  of  the  WLSRE  technique.  The  program 

2 

estimates  the  variance  (ov  )  of  the  input  data  baaed  on  third  order 

2 

ordinary  diffsrances  and  compares  it  to  the  variance  (o^  )  of  the 

input  data  baaed  on  deviations  from  the  modal.  Based  on  this  test 
the  span  slse  will  be  increased  or  decreased.  The  span  else  can 
also  be  controlled  externally  if  the  user  specifies  the  ewlmum 
slse  of  the  span  that  he  desires.  When  the  program  has  increased 
the  span  else  such  that  it  has  raached  the  specified  maximum,  them 
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it  will  continue  at  this  eoastsnt  spsn  slss  until  tht  aodsl  no 
longsr  fits  ths  dsts  sad  ths  prograa  forcss  th«  sst  slss  to  dserssse. 


Initially  va  dataralna  tha  atata  aatiaataa  X,  X,  aodX  Item  laaat 
aquaraa  aquation 


X  -  (H^f1  H*Z. 


(1) 


Uainq  X  froa  lq.  (1)  and,  +,  tha  tranaltion  matrix,  wa  saa  that 


Xtfl  “  «f 


U) 


Mow  wa  corract  tha  pradlctlon,  X,  with  tha  corraction 


c  •  (GfaSra)-1  Rtw(z-HX)> 


0) 


Eq.  (3)  raducaa  to 


C  -  P(K,  At)  HC  (Z-HX) 

wharo  P  dapanda  only  on  X  (.pan  also)  and  At  (lncraaant  of  tlao 
batwaan  observations), 

tha  datallad  computation  of,  Eq.  (4)  1.  conaidorad  in  tha  Appandix. 
Tha  corrected  atata  bacoaaa 


To  obtain  the  r endow  nolo#  content  of  the  obeervatlona ,  uee  the 
equation 


2  1(0  -  * 
_  ■  • _ X  X  * 


fe*  ‘  ■  Ife).  t  -  ^u\ . 


(5) 


where 


-  tx  -3Z1_1  +  3Z1-2  -  Z^, 

2 

ov  ■  variance  of  obeervatlona, 
m  ■  order  of  differaneea  (3rd), 

K  -  open  aiae, 

2 

The  variance  oc  of  the  new  apan  (prevloue  epan  plue  or  nlnua  appropriate 
obeervatlona)  takea  tha  fora 


Jc  "FI 


where, 


r  ez1  i 


r  •  EZt  -  (EZ1  SZti  EZji?)  P  EZ^ 


EZ.l* 


!  I 


.  J 


K  >  9 
i  ■  t 


( K-3 ^  „  fiwO  .  ,  4+1  ' 

-  \-r}  *  -{-rj . *  ♦  (77 


(6) 
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a  2 

Taking  cha  ratio  of  ®c  Bq.  (6)  and  oy  Eq.  (5).  wa  obtain 


If 

R  <  <K-3.  K-3) 

than  wa  aecapt  tha  corraetad  atata  aatimata  aa  balng  optimal  and  incraaaa 
K  by  2  and  naka  tha  naxt  pradletlon  with  Bq.  (2). 

Howwvar  if 


R  >  F99  (K-3,  K-3) 

than  wa  can  aaauna  that  probably  tha  nodal  (2nd  dagraa  polynomial  In  thla 
caaa)  no  longar  flta  tha  Input  data,  and  daeraasa  tha  apan  alia  by  dalatlng 
tha  approprlata  obnarvatlon(a)  fron  tha  apan  that  haa  fallad  tha  F-taat. 

In  thla  caaa,  wa  nuat  naka  anothar  corractlon  with  Bq.  (3)  bafora 
obtaining  tha  naxt  corraetad  atata. 

Obvloualy 


X  ■  X  +  c 
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yielde  the  noothut  state  estimates.  To  obtain  tha  filter  ad  atata  estimates, 
«•  have  only  to  propagate  tha  smoothed  atata  aatinataa  to  tha  and  point. 


> 

i 


r 


i 


a 
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XXX.  PERFORMANCE  OF  FIXED  LEAST  SQUARES  AMD  DYNOSS 
A.  Minim  Envl temtzl 
1.  Fixed  luit  Squares 

Tha  position  eatimatea  generated  with  Fixed  L8-51  at* 

aecaptsbla  for  tha  Athana  trajectory.  Figures  1  and  A  do  not 

show  any  significant  differences  se  this  scale  between  tha  standard 

positions  and  those  ganaratsd  with  LS-51. 

On  tha  Athana  data,  we  note  that  tha  fixed  LS-51  velocity 

as t las tea  (Fig.  5)  are  eloaa  to  the  standard  (Fig.  2)  during 

Intervals  of  tins  whsn  there  Is  little  or  no  change  in  acceleration. 

However,  this  Is  not  tha  case  during  abrupt  changes  In  acceleration. 

In  Fig.  5  note  that  during  second  stage  ignition  (approx.  46. 0  sec), 

fixed  L8-51  la  In  error  by  about  55  ft/aac.  During  second  stage 

burnout  (approx.  56.0  sac),  the  error  la  not  as  pronounced  as  before 

since  the  change  in  acceleration  Is  aore  gradual  than  during  Ignition 

Tha  error  In  this  case  la  around  30  ft/sac.  The  above  observations 

can  clearly  ba  saen  In  the  enlarged  graphs  (Figs.  6  and  7). 

Examination  of  the  Athana  data  shows  that  tha  fixed  LS-51 

acceleration  estimates  (Fig.  8)  are  acceptable  during  non-event 

periods.  Howavar  during  events,  ws  note  that  tha  maxlnun  error  Is 

2 

approximately  120  ft/sac  (See  the  enlarged  graphs,  Fig.  9  and  10). 

2.  0TMO8S 

Tha  DYMOSS  ganaratsd  position  estimates  for  tha  Athana 
are  extremely  doss  to  tha  standard.  Compere  Fig.  4s  to  Fig.  1. 
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On  tha  Atbana  velocity  estimates,  not  only  doaa  DYWOSS 
•how  definite  laproveaant  ovar  L8-51,  but  tha  niaga  error  la 
•bout  9  f t/aac.  This  arror  la  noted  only  Cor  ob«  time  point  during 
Ignition.  Otherwise,  tha  arror  la  significantly  laaa  than  9  ft/aae. 
This  nan  olaarly  ba  aaan  In  Fig.  6.  Tha  excellent  performance  nan 
ba  attrlbutad  to  tha  daeraaaa  In  tha  apan  alia.  Fig.  8  at  bottom 
•howa  time  va.  apan  alia. 

Coaparlaon  of  DYNOSS  ganaratad  accalaratlon  estimate* 

(Fig.  8)  with  tha  standard  (Fig.  3),  va  notlca  close  agraaaant  at 
all  tlmaa.  Ha  can  alao  aaa  from  Flga.  9  and  10,  tha  auparlor  par- 
fonanea  of  DTMOSS  during  avanta. 

B.  Molar  favlronmant 

1.  Fix ad  Laaat  Squares 

Tha  random  nolao  lntroducad  Into  tha  Athana  position  data  waa 
30  fast,  wharaaa  tha  nolaa  lntroducad  Into  tha  High  Accalaratlon  data 
waa  1  foot. 

Although  tha  Athana  poaitlon  aatlaatca  obtained  from  LS-51 
do  poaaaaa  error,  they  are  accaptabla.  Similarly  tha  LS-61  poaitlon 
aatlaataa  on  tha  High  Acceleration  data  era  accaptabla.  Compare 
Fig.  11  to  Fig.  1  and  Fig.  17  to  Fig.  14. 

With  tha  Introduction  of  50  faat  of  nolaa  to  tha  Athana 
Standard  positions,  va  notice  in  Fig.  12  that  tha  velocity  aatlaataa 
ganaratad  by  LS-51  have  an  oscillating  pattern  throughout  tha  flight. 
Tha  error  la  anywhere  between  0  and  75  ft/ sac.  The  nature  of  thla 

kind  of  arror  usually  cannot  ba  tolerated,  if  a  high  velocity  accuracy 


Is  required  for  impact  prediction. 

The  LS-61  velocity  estiaataa  on  the  1  foot  nolae  High 
Acceleration  date  (Pig.  18)  dsnonatrataa  that  during  periode  when 
the  acceleration  la  not  conatanti  there  la  a  biaa  in  the  data. 

The  aagnltuda  and  algo  of  the  velocity  bias  depends  on  the  rate  of 
the  acceleration  change  and  lta  elgn  < compare  LS-61  Pig.  18  with 
Pig.  IS). 

The  undesirable  pattern  of  the  L8-51  acceleration  ea tine tea 
on  the  Athena  for  high  noise  levels ,  nakas  it  unacceptable.  The 
"randon"  acceleration  astlaatss"  generated  by  L8-51  are  shown  in 
Pig.  13. 

Pigurs  19  showa  the  acceleration  estimates  propagated  for 
the  High  Acceleration  data  corrupted  with  1  foot  randon  error.  Mote 
that  the  true  mlnlnuma  and  naxinuns  are  never  achieved  with  a  fixed 
span  site. 

2.  DYNOSS 

The  position  data  genaratad  by  DYNOSS  for  both  tra¬ 
jectories  do  possess  slight  errors  but  they  are  batter  than  those 
from  fixed  least  squares. 

Conpariaon  of  DYNOSS  to  LS-51  (Fig.  12)  velocity  sstl- 
matas  indicates  that  DYNOSS  is  better.  The  effect  noted  in  the 
LS-S1  case  has  bean  avoided  since  DYNOSS  has  increasad  the  span  else 
to  the  naxinun  specified  (See  botton  Fig.  13).  The  nain  reason  for 
the  Increase  in  span  else  can  be  attributed  to  the  high  noise  content 
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of  Che  fit!  (SO  ft).  Also  In  Fig.  15,  vs  not*  a  alight  decrease  In 
apes  alsa  tfua  to  aaeond  stage  Ignition  and  burnout. 

Flgura  18  reveals  that  tha  DTNOSS.  velocity  aatlnataa  generated 
for  tha  High  Acceleration  data  with  1  foot  of  naiaa  conparo  favorably 
with  tha  ataadard  (fig.  15).  Special  nota  ahould  ha  aada  that  tha 
biaa  obtalnad  with  18-61  haa  baan  rtawni  by  using  tha  optimal  apaa 
alia. 

Tha  uaa  of  DTNOSS  to  ganarata  Athana  accalaratlon  aatlnataa 
(fig.  13)  in  a  high  noiaa  anvlronaant  haa  provan  axtraaaly  halpful. 
Kota  that  tha  harmful  af fact  of  uaing  18-51  on  high  noiaa  laval 
data  to  obtain  accalaratlon  aatlnataa  haa  baan  avoidad.  Conparlaon 
of  DVM088  to  18-51  (fig.  13)  ahowa  tha  ranarkahla  laprovanant  In 
flltar  output  whan  tha  apan  alaa  la  allowad  to  Incraaao  for  non - 
avant  high  noiaa  laval  data. 

Tha  harmful  affact  of  uaing  too  long  a  apan  (18-61)  on 
low  noiaa  (1  Ft)  data  to  ganarata  accalaratlon  aatlnataa  can  ba 
aaan  In  rig.  19.  Nota  tha  DTNOSS  (Fig.  19)  ganaratad  data  la  vary 
cloaa  to  tha  standard  (Fig,  16). 

C.  Connutinn  Efficiency 

Figures  20  and  21  show  a  naaeure  of  tha  eoeputer  time  required 
to  generate  DTNOSS  and  L.S,  respectively.  Tha  tinea  Indicated  show 
tha  tine  required  to  Input,  compute,  and  output  one  anoothed  point. 

In  Fig.  ZO  maximum  span  naans  that  tha  span  was  allowad  (through  load 
card  options)  to  vary  (depending  on  noise  and  events)  between  9  polnta 
and  tha  glvan  maximum. 
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LEAST  SQ  COMPUTING  TIME 
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Using  a  UNIVAC  1108  Computer 


IV.  BIBMAKY 


vv  >■• 


For  noiM  free  (except  for  round-off  orrora)  data,  DYNOSS  was 
|  abla  to  provida  batter  state  eatlmatea  than  LS-51.  Thia  la  readily 

'  apparent  during  aventa  In  tha  enlarged  velocity  and  acceleration 

I 

graphe  (Fig  6,  7,  9,.  and  10)  . 

if 

With  the  Introduction  of  SO  feat  and  1  foot  of  nolae  to  the 

4- 

Athena  and  High  Acceleration  data  respectively,  we  noted  DYNOS8 
performance  waa  extremely  good  (Fig.  12  and  13).  DYN0S8  proved 
Invaluable,  eapedally  for  high  accelaration  non-event  (not  com¬ 
pletely  evantleaa,  but  for  gradual  throating  perloda)  data  when  a 
long  epan  craataa  anundaeirabla  mleflt  in  the  L8-S1  velocity  estl- 
aatea  and  acceleration  aatimatee  (See  Fig.  16  and  19). 

In  our  lnvaatlgatlnn,  we  alao  noticed  that  the  OYNOSS  apan 
aelector  for  nolae  free  event  data  did  In  fact  decreaee  the  apan 
alaa  aa  expected  during  eventa  (See  botton  of  Fig.  8)  and  therefore 
propagated  optimal  atate  eatimatea  (Fig  6,  7,  9  and  10).  Fig.  13 
at  the  bottom  ehowa  the  ability  of  DYNOSS  to  adjust  to  a  nolay 
environment. 

In  Flga.  20  and  21  we  noticed  that  DYNOSS  la  quite  efficient 
and  that  tha  computer  time  ie  Invariant  ylth  length  of  apan. 

We  recognise  that  a  considerably  lower  apan  else  could  have 
been  used  where  nolae  content  la  low.  Certainly  thia  would  have 
ahown  Improvement  In  the  LS  velocity  estimates  (Fig.  IS)  from 
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1  to  2.33  see.  Howavar,  tha  aatlnataa  generated  from  2.33  aac  to 
3.00  would  than  hava  deteriorated.  Nuaaroua  othar  axanplaa  could  ba 
cltad,  but  tha  lnportant  point  ia  that  fixed  Laaat  Square*  nitht 
propagata  optimal  a tat*  aatlnataa  only  at  certain  tinea,  wheraaa, 
DYM088  alwaya  propagataa  optimal  atata  aatlnataa. 

Wa  hope  that  wa  hava  aucceedad  in  giving  adequate  axanplaa  to 
point  out  tha  advantagaa  afforded  by  DYNOSS. 
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V.  CONCLUSIONS 

On*  of  the  major  conclusion*  reached  1*  that  dynamic  filtering/ 
smoothing  is  Indispensable. 

In  a  data  reduction  complex  Ilka  Data  Reduction  Division  at 
WSMR,  New  Mexico,  0YNOS8  Is  essential.  Tha  major  reason  Is  the 
diversity  and  enormous  volume  of  data  processed,  frequently  do  not 
allow  tha  data  analyst  to  judiciously  select  the  span  sise  required 
to  filter  data.  However,  since  acceleration  and  noise  are  non-* 
stationary;  not  even  the  best  analyst  can  select  one  span  sise  to 
acquire  optimal  etate  estimates  throughout  a  complete  missile  flight. 

Thus,  only  by  utilising  the  optimal  span  sise  in  any  given 
time  lntervel  can  a  filter  provide  a  true  description  of  the 
missile  dynamics. 

In  view  of  all  the  advantages  afforded  by  DYNOSS  and  tha  test 
requirements  for  development  of  high  accelerated  multi-stage  missiles, 
the  implementation  of  dynamic  filtering  ie  not  only  neceseery  but 
mandatory. 
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APPENDIX 


The  purpoaa  of  thla  appendix  la  to  provide  detailed  computation  of 
Eq.  (4).  Our  fir at  teak  la  the  coaputatlon  of  H.  However  H  takea  on 
different  veluea  depending  on  whether  our  apan  aixe  ia  conatent,  increasing, 
or  decreaalng.  Whan  K  la  lncraaaad  bp  two  obaervatlone,  we  have 


(K-3)At 

i 

1 

2 

Vw  2 

1 

sit. 

i2  /  i 

1 

L 

2 

L  2  J 

/  2J 

Decraaalng  K  yialda  two  valuaa  for  H. 

Flrat  if  K  la  decreaaed  by  deleting  two  obaervationa  at  the  beginning 
of  a  apan,  we  have 


i  1 

_ 


-(K-fl)At 

2 

■(K+3)At 

2 


V' 
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Second,  if  K  ie  decreased  by  delating  two  observations  one  at  tha 
beginning  and  tha  other  at  tha  and  of  a  span  yields 


Tha  procedure  to  Maintain  a  constant  apan  nakea  use  of  two  Iterations  - 
of  the  basic  WLSRK  equations.  During  the  first  iteration ,  we  increase 
the  given  apan  by  adding  an  obaervatlon  at  the  end.  The  second  iteration 
will  then  decrease  the  apan  by  deleting  one  observation  at  the  beginning. 
In  the  foraer  caae,  H  la  computed  an  follows 
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In  thn  Uttar  cm* 


1.0 


zsz*n± i 

2 


Fron  th*  basic  aquation  a  •  (G  +  HCWH)  * ,  cz  can  ba  fact or ad  from  G 

and  W,  l**ving  (G  +  HtWH)”1  a*  a  function  of  K  and  At.  Thua  tha  <G  +  H*  WH) 
matrix  dots  not  hav*  to  ba  lnvartad  every  tine  but  can  ba  predicted 
based  on  th*  known  valuaa  of  K  and  At.  It  can  ba  ehovn  that 
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Once  again  w*  emphaalae  that  K(  la  the  K  that  takes  into  consideration 
tha  obaervatlon  matrix  Z. 


Therefor#,  tha  correction  taken  the  form 
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In  tha  abova  aquation  if  we  aubatitute 


for  W 


we  obtain 


3  (3K  -7) 
4K(K  -4) 


-30 


-30 

- Z - 7 

K(K  -4)At 


12 


K(K4-l)At 


K(K  -4)At 


ArP* - 

^At^/K(K  -1)  (K  -4) 


H  (Z-HX) 


UraiNCEB 


1‘  Mi«.«ChS1!0n,C,»  ,'Co^*ri»oo  o{  Static  and 

5£».  !JiS2i.  £$:,1,i5J.c"'“Mtl“  . 

2.  Coaatock  0.,  Wright  M.,  Tipton  V.,  "Handbook  of  Dat* 


AN  "OPTIMIZER”  FOR  USE  IN  COMPUTER 
SIMULATION!  STULTES  WITH  A  PROTOTYPE  (U) 
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ABSTRACT. 

Bacauae  of  thalr  extrema  complexity,  many  military  operatlona  reaaarch 
problema  cannot  be  examined  analytically,  but  inataad  muat  be  examined  by 
meana  of  computer  eimulation.  However,  the  uaa  of  aimulation  In  attempting 
to  locate  an  optimum  solution  often  degenerates  into  a  trial-and-error 
process .  This  paper  describes  a  study  aimed  at  providing  the  aimulation 
user  with  efficient  mathoda  of  searching  for  optimum  computer  aimulation 
solutions. 

This  study,  to  date,  has  resulted  in  the  following  three  major 
accompllahmenta  t 

(1)  Development  of  a  prototype  "OPTIMIZER"  computer  program 
consisting  of  a  set  of  search  techniques. 

(2)  Empirically  derived  information  to  aid  in  the  selection 
of  a  search  technique, 

(3)  Successful  application  of  the  prototype  "OPTIMIZER"  to  an 
existing  computer  simulation. 

Each  of  theae  accomplishments  is  discussed  in  this  report. 
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I.  INTRODUCTION 


Became  of  their  extreme  complexity,  many  military  operations  research 
problems  cannot  be  examined  analytically,  hut  instead  must  be  examined  by 
means  of  computer  simulation.  When  computer  simulation  is  involved,  an 
optimum  solution  is  not  obtained  by  analytic  techniques,  but  rather  by  a  search 
of  the  relevant  parameter  space.  In  general,  this  search  for  an  optimum 
solution  rapidly  degenerates  Into  a  trial-and-error  process.  The  goal  of  the 
present  study  is  the  development  of  more  efficient  procedures  for  obtaining 
optimum  computer  simulation  solutions. 

The  introductory  section  of  this  report  provides  the  necessary  background 
information  and  summarises  the  progress  made  toward  the  indicated  goal.  The 
remaining  sections  present  the  details  of  the  various  aspects  of  the  current 
study  effort. 

A.  BACKGROUND 

A  computer  simulation  may  quite  realistically  be  regarded  as  involvings 
"black  box"  in  which  the  values  of  input  variables  or  parameters  are  combined 
in  some  manner  to  produce  output  variables  or  responses.  These  responses 
are  visually  figures  of  merit  of  some  type.  Figure  1  summarises  this  "black 
box"  view  of  computer  simulation  when  only  one  output  variable  (response)  is 
involved. 

As  will  be  noted  from  this  figure,  the  input  variables  are  of  two  types: 

(l)  controllable  factors  and  (2)  uncontrollable  factors.  The  controllable  factors 
are  those  input  variables  having  values  which  can  be  controlled  in  the  "real 
world."  For  example,  input  variables  such  as  submarine  speed,  aircraft  alti¬ 
tude,  and  destroyer  bearing  pertaining  to  United  States  Naval  forces  would  be 
controllable  factors  because  their  values  could  be  varied  at  will  by  the  appro¬ 
priate  decision  maker  in  the  Naval  chain  of  command.  On  the  other  hand, 
these  input  variables  (submarine  speed,  aircraft  altitude,  and  destroyer  bearing) 
relating  to  enemy  forces  must  be  regarded  as  uncontrollable  factors  because 
the  U.S.  Naw  cannot  vary  the  values  of  these  factors  at  will.  In  addition, 
environmental  characteristics  such  as  sea  state,  temperature,  and  amount  of 
precipitation  must  also  be  regarded  as  uncontrollable  factors  because  their 
values,  too,  cannot  be  controlled. 
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A»  Figure  1  indicates,  the  computer  aimulation  "black  box"  produce*  the 
observed  response  y  which  is  a  function  of  both  the  controllable  and  uncontrollable 
factors.  In  probabilistic  computer  simulations  where  random  variables  are  in¬ 
volved,  the  observed  response  y  may  vary  even  if  two  identical  situations  are 
run,  i.e. ,  if  all  input  variables  have  the  same  values  both  times  the  simulation 
is  used  to  produce  a  response.  *  In  this  situation,  the  observed  response  y  may 
be  assumed  to  vary  about  a  true  response  g  (Xj ,  . ,  . ,  X^  |  Z  j ,  . . . ,  Z^),  which, 
in  statistical  terms,  is  an  expected  value.  The  variation  of  y  from  this  true 
response  may  be  regarded  as  sampling  error  which  arises  from  a  random  vari¬ 
able  <  having  zero  expected  value  but  positive  variance. ^  Thus  the  observed 
response  is  composed  of  two  quantities  and  may  be  expressed  as 

y  s  g  (Xj . j  Zj .....  Z^)  +  «,  the  form  presented  in  Figure  1 ,  It  should 

be  noted  that  the  true  response,  g,  defines  a  surface  in  k -dimensional  space. 

This  surface  is  known  as  a  response  surface. 

The  optimum  solution  con  gists  of  those  values  of  the  controllable  factors 
which  produce  the  true  optimum  response,  i.e.,  the  optimum  expected  value. 

It  should  be  noted  at  this  point  that  optimisation,  per  se,  in  simulation  is  an 
ultimate  goal  which  in  practical  situations  is  usually  unreachable.  Ordinarily, 
the  most  that  one  can  hope  to  accomplish  given  the  constraints  of  limited  time 
and  funds  is  the  Location  of  an  improved,  or  a  "good"  solution.  That  is,  the  re¬ 
sult  may  be  the  identification  of  a  solution  which,  while  not  necessarily  a  global 
optimum,  is  acceptable  and  better  than  any  previously  available  solution.  To 
emphasize  this  fact  in  the  current  study,  the  term  "optimization"  when  placed 
in  quotation  marks  refers  to  the  attempt  to  locate  the  true  optimum  solution. 

Thus,  the  "optimum"  solution  which  is  found  by  "optimization"  may  not  be  the 
true  optimum  solution. 

B.  SUMMARY  OF  FIRST  YEAR'S  EFFORT 

HRB-Singer,  Inc, ,  under  a  contract3  sponsored  by  the  Office  of  Naval 
Research,  has  focused  on  "optimization"  within  that  class  of  simulations  where 

*It  is  assumed,  of  course,  that  different  sequencos  of  pseudo-random  numbers 
are  used  in  each  computer  run. 

2If  the  simulation  is  deterministic,  however,  «  is  identically  zero. 

Contract  No.  N00014-69-C-0285. 
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a  single  response  (i.  e.,  figure  of  merit)  exists,  the  controllable  factors  are 
continuous  (or  at  least  approximately  so)  and  there  are  no  constraint*  on  the 
controllable  factors. 

An  HRB-Singer  report*,  which  was  published  in  July  1970,  describes  the 
progress  made  during  the  first  year  of  the  ONR  contract.  This  report,  which 
discusses  two  categories  of  methods  which  may  be  used  to  aid  in  "optimisation", 
concludes  that  the  largest  potential  payoff  would  come  from  the  development 
of  an  "OPTIMIZER"  computer  program.  This  "OPTIMIZER",  which  could  be 
coupled  to  existing  simulations,  would  automate,  to  a  degree,  the  "optimisation" 
process. 


1.  Methods  to  Aid  in  "Optimization" 

Methods  to  aid  in  the  quest  for  an  "optimum"  solution  may  be  classified 
into  two  broad  categories  :  (l)  internal  methods  and  (2)  external  methods. 

Internal  methods  involve  tinkering  with  the  inner  workings  (the  mathematical 
relationships  and  the  computer  programming)  of  the  "blackbox".  Because  internal 
methods  are  actually  built  into  the  simulation  when  the  mathematical  models 
are  constructed,  the  analysts  and  modelers  responsible  for  model  development 
must  Incorporate  these  methods  directly  into  the  "black  box".  External  methods, 
on  the  other  hand,  do  not  affect  the  Inner  workings  of  the  "black  box",  and  are 
essentially  independent  of  model  construction.  These  methods  specify  search 
techniques  which  involve  experimentation  with  different  values  of  the  controllable 
factors,  usually  using  the  output  of  the  "black  box"  as  feedback. 

2.  The  Concept  of  an  "OPTIMIZER"  Computer  Program 

Smith (1970a)  concluded  that  search  techniques  tend  to  provide  the  most 
useful  means  of  "optimization",  and  that  the  existence  of  these  techniques  in  an 
"OPTIMIZER"  computer  program  should  prove  of  value  to  the  computer  simula¬ 
tion  user.  Such  an  "OPTIMIZER"  program  could  be  constructed  independently 

I 

of  any  specific  computer  simulation,  and  would  thus  be  general  enough  to  provide 
flexibility  and  wide  applicability.  In  addition,  the  user  would  not  need  an  extensive 

1  Smith  (1970a). 

2  f 
Methods  from  each  of  these  categories  are  discussed  in  Smith  (i970a). 
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knowledge  of  the  mathematics  underlying  the  techniques  since  all  computations 
could  be  handled  within  the  structure  of  the  "OPTIMIZER", 

The  "OPTIMIZER"  would  have  to  be  designed  in  such  a  manner  that  it 
could  be  granted  access  to  the  inputs  and  outputs  of  the  computer  simulation,  as 
well  as  to  additional  data1  which  the  simulation  user  would  supply.  Figure  2  sum¬ 
marizes  the  design  concept  defining  the  ultimate  goal  of  the  current  study,  namely 
the  eventual  construction  of  a  general  "OPTIMIZER"  program  which  can  be 
tailored  to  any  specific  computer  simulation  situation  by  additional  information 
supplied  as  input  by  the  user. 

In  essence,  the  "OPTIMIZER"  would  obviate  the  need  for  an  expert  who 
is  familiar  with  the  various  search  techniques  and  the  underlying  mathematics, 
because  they  would  be  contained  within  the  "OPTIMIZER"  itself.  As  indicated  in 
Figure  2,  the  simulation  user  would  be  able  to  communicate  to  the  "OPTIMIZER" 
information  in  addition  to  the  standard  simulation  input.  This  information  would 
be  used  by  the  "OPTIMIZER"  to  determine  the  best  search  technique  to  be  adopted 
for  the  specific  simulation  situation.  Thus,  the  "OPTIMIZER"as  visualized  would 
have  a  double  optimization  task.  Not  only  would  it  attempt  to  find  an  "optimum" 
solution,  but  it  would  also  attempt  to  determine  the  most  appropriate  "optimising" 
technique  to  be  used  in  any  situation. 

C.  SUMMARY  OF  CURRENT  STUDY  EFFORT 

During  the  past  year,  a  prototype  "OPTIMIZER"  computer  program  has 
been  programmed  and  checked  for  errors  by  running  a  number  of  test  cases. 

This  "OPTIMIZER"  consists  of  seven  specific  search  techniques.  At  the  present 
time,  selection  of  the  particular  technique  to  be  applied  in  any  given  simulation 
situation  must  be  made  by  the  user  of  "OPTIMIZER".  However,  it  is  planned 
that  the  internal  logic  of  the  "OPTIMIZER"  will  eventually  include  decision 
criteria  to  permit  automatic  selection  of  the  best  technique  for  the  existing 
simulation,  based  on  user -supplied  information  about  the  simulation 
characteristics.  To  provide  such  decision  criteria,  an  empirical  study  of 
the  search  techniques  has  been  conducted  using  known  response  surfaces 


1  Such  as  the  starting  point  for  the  search  and  the  number  of  computer  runs 
which  are  available, 
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Fit.  2  KTIMTC  imCTIVE  Of 


which  were  constructed  to  have  various  characteristics.  In  addition  to  these 
empirical  studies,  the  prototype  "OPTIMIZER"  was  applied  to  an  existing  com¬ 
puter  simulation  (i.e.,  to  an  unknown  response  surface). 

Thus,  the  current  year' s  effort  consisted  of  three  phases: 

Phase:!:  Construction  of  a  Prototype  "OPTIMIZER" 

Phase  II:  Empirical  Examination  of  the  Performance  of  the  Search 

Techniques  on  Constructed  Response  Surfaces 

Phase  III:  Application  of  the  Prototype  "OPTIMIZER"  to  an  Existing 
Computer  Simulation 

Each  of  these  three  phases  is  discussed  in  the  following  sections. 

1 .  The  Prototype  "OPTIMIZER" 

The  prototype  "OPTIMIZER"  was  constructed  to  function  as  an  executive 
program.  As  such,  the  "OPTIMIZER"  provides  linkages  with  the  simulation, 
carries  out  the  required  bookkeeping,  and  performs  the  mathematical  calculations 
connected  with  the  search  technique  being  used.  In  such  a  manner,  the 
"OPTIMIZER"  maintains  control  over  the  values  of  the  controllable  factors  to 
be  used  at  each  step  in  the  search  for  an  "optimum". 

The  "OPTIMIZER"  contains  a  total  of  seven  search  techniques,  whioh 
are  variations  of  three  basic  techniques.  The  first  technique,  Random  Search, 
simply  selects  at  random  the  values  of  the  controllable  factors  to  be  used  in 
each  simulation  run.  The  second  technique.  Single-Factor,  examines  one  factor 
at  a  time  by  means  of  coordinated  movement  of  a  given  factor  while  all  others 
arc  held  constant.  The  third  technique,  Response  Surface  Methodology  (RSM), 
permits  the  examination  of  all  factors  simultaneously  by  means  of  an  experimental 
design  from  which  the  estimated  gradient  direction  may  be  determined. 

2.  Empirical  Studies 

As  mentioned  previously,  the  specific  search  technique  to  be  applied 
must  be  selected  by  the  user.  In  order  for  the  user  (or  eventually,  the 
"OPTIMIZER"  logic)  to  choose  the  best  technique,  the  performance  of  the 
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techniques  must  be  evaluated.  Because  no  mathematical  basis  exists  for  such 
an  evaluation  relative  to  characteristics  which  may  vary  from  simulation  to 
simulation,  it  is  necessary  to  perform  empirical  studios  in  order  to  obtain  the 
required  information.  A  major  portion  of  the  current  study  was  devoted  to  the 
construction  of  a  framework  for  an  empirical  evaluation  of  the  performance  of  the 
search  techniques  for  different  categories  of  simulation  situations. 

The  evaluation  itself  was  based  on  studies  using  response  surfaces  con¬ 
structed  from  known  mathematical  relationships  which  were  specifically  chosen 
tor  the  investigation.  The  mathematical  relationships  generating  the  response 
were  so  constructed  that  the  true  optimum  was  known,  providing  comparison  of 
the  performance  of  each  technique  with  the  true  optimum  as  well  as  comparison 
with  the  performance  of  other  techniques.  Thu  performance  of  a  technique,  of 
course,  is  directly  related  to  the  "optimum"  found  by  that  technique.  Specifically, 
the  performance  of  a  given  search  technique  was  measured  in  -.erms  ol  the 
relative  gain  to  be  expected  from  using  that  technique,  where  relative  gain  was 
defined  as: 


f  [ 

True  response  at  "optimum"  point"] 
found  by  search  technique  j 

' 

[True  response  at"] 

Lsta  rting  pr  !  r.t  J 

( ' 

jl’ruo  optimum  response  J 

^True  response  at  starting  poiritj 

Each  of  the  search  techniques  was  evaluated  for  a  variety  of  simulation 


characteristics  in  the  situation  where  the  ava,,nble  number  of  computer  runs 
was  relatively  small  tompared  with  the  number  of  controllable  factors.  Specific 
characteristics  considered  were. 

1.  The  number  of  controllable  factors. 

2.  The  number  ol  computer  runs  available. 

1.  Distance  from  starting  point  to  true  optimum  point 

4.  Relative  size  of  the  random  error  associated  with  the  response  sur tac e  . 

5.  Relative  activity  of  the  controllable  factors. 
f>.  The  presence  (or  absent  e)  of  interaction, 

7  The  presence  (or  absente)  ol  lot.al  optima. 
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Computer  routines  carried  out  the  evaluation  and  provided  summary 
tables  and  statistics  of  the  performance  of  each  of  the  techniques.  As  a  result  of 
this  evaluation  task,  empirically  derived  payoff  matrices  for  use  in  the  selection 
of  a  search  technique  exist.  These  matrices,  together  with  any  existing  knowledge 
of  the  simulation  situation  under  consideration,  permit  the  user  to  make  a 
relatively  efficient  choice  of  a  search  technique. 


3 .  Application  of  the  "OPTIMIZER1 ' 

In  addition  to  the  research-oriented  phase  of  the  current  study,  HRB- 
Singer,  with  the  cooperation  of  the  U.  S.  Army  Strategy  and  Tactics  Analysis 
Group  (STAG),  applied  the  prototype  "OPTIMIZER"  to  a  computer  simulation 
known  as  FORECAST  II,  which  was  developed  by  STAG.  Although  interface 
procedures  were,  in  general,  easily  carried  out*,  difficulties  encountered  with 
the  controllable  factors  indicated  that  assumptions  made  for  the  prototype. 
"OPTIMIZER"  are  not  totally  acceptable  for  many  large  military  models. 

In  particular,  the  variables  chosen  as  controllable  factors  in  FORECAST  II 
were  neither  continuous  nor  unconstrained,  thus  violating  two  assumptions  on 
which  the  "OPTIMIZER"  was  developed.  In  an  attempt  tu  bypass  the  latter 
difficulty,  a  transformation  of  the  controllable  factors  was  used  to  alter  the 
"optimization"  problem  into  one  which  was  unconstrained.  Because  the  integer 
values  which  might  be  assumed  by  the  controllable  factors  were  not  extremely 
small,  it  was  decided  to  proceed  by  treating  these  factors  as  approximately 
continuous. 


Results  of  the  application  proved  positive.  With  only  a  small  number 
of  "OPTIMIZER"  runs,  definite  improvement  was  achieved.  This  represented 
an  increase  in  the  observed  figure  of  merit  from  .  397  to  ,  436.  Because  a 
theoretical  upper  bound  to  the  figure  of  merit  is  .  500,  the  relative  gain 

. 436  -  . 397 

provided  by  the  "OPTIMIZER"  may  be  calculated  to  be  at  least  — — — — — 

.  500  -  ,  397 


or  approximately  38%. 


A  fairly  detailed  account  of  the  application  of  the  "OPTIMIZER"  to  FORECAST  II 
is  contained  in  the  Memorandum  for  Record  of  25  February  1971  and  6  April  1971 
prepniod  by  Captain  D.  I,.  Renfro  (forme  rlv  with  Systems  Development  Division 
STAG). 
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ii.  mi-:  prototype:  "optimizer" 


The  prototype  "OPTIMIZER ",  which  is  comprised  of  a  set  of  search  tee'n- 
nif|ues,  was  developed  to  function  ns  a  monitor  or  executive  program.  (As 
previously  mentioned,  the  search  technique  to  he  used  in  any  particular  case 
must  lie  specified  by  the  user.)  The  linkage  between  "OPTIMIZER"  arid 
the  simulation  is  via  a  FORTRAN  CALI,  statement,  with  data  exchanged  via 
the  CALI,  statement  and  COMMON  statements. 

The  "OPTIMIZER"  reads  the  initial  (input)  values  for  the  control  (able 
factors  together  with  any  other  required  input  data,  anti  then,  in  general,  using 
observed  response  feedback,  changes  the  values  of  the  controllable  factors 
according  to  the  algorithms  inherent  in  the  prescribed  search  technique. 

In  addition,  the  "OPTIMIZER"  does  the  necessary  accounting,  maintaining  a 
record  of  required  , ufor mation  such  as  the  number  of  runs  executed  and  remaining, 
tile  results  nf  calculations  required  by  the  search  techniques,  the  best  response 
observed,  and  the  values  of  the  controllable  factors  which  have  resulted  in  this 
best  response. 

The  seven  search  techniques  in  the  "OPTIMIZER"  are  based  on  three 
principal  techniques  :  (a)  Random  Search,  (b)  Single- Factor  approach,  and 
(e)  Response  Surface  Methodology  (RSM).  Of  these  three  techniques,  Random 
Search  is  a  nonaria ptive  search  technique,  while  the  Single- Fa ctor  approach 
and  RSM  are  udapti  •  search  techniques.^ 

The  prototype  "OPTIMIZER"  consists  of  the  following  seven  specific  search 
techniques : 


These  basic,  search  techniques  are  described  in  Smith  (1970a).  Another  tech¬ 
nique,  the  factorial  design,  was  not  included  in  the  prototype  "OPTIMIZER" 
because  of  the  unwieidly  number  of  computer  runs  it  requires  for  even  a  small 
number  of  factors, 

^In  a  nonadaptive  technique,  the  observed  response  from  a  computer  run  does 
not  influence  the  values  of  the  controllable  factors  chosen  for  any  succeeding 
run.  An  adaptive  technique,  however,  bases  selection  of  the  values  of  con¬ 
trollable  factors  for  future  computer  runs  on  the  observed  responses  from 
previous  runs. 
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(1)  Random  Search 

(2)  Single-Factor  Approach 

(3)  Single-Factor  Approach  with  Acceleration 

(4)  RSM  -  Variation  I 

(5)  RSM  -  Variation  I  with  Acceleration 

(6)  RSM  -  Variation  II 

(7)  RSM  -  Variation  II  with  Acceleration 


The  following  sections  provide  descriptions  of  the  basic  search  techniques  and 
the  variations  as  they  exist  in  the  prototype  "OPTIMIZER",  and  specify  the  input 
data  required  of  the  user.  By  far  the  most  space  is  devoted  to  RSM  because  of 
its  relative  complexity.  IL  will  be  assumed  in  the  following  descriptions  that 
there  are  k  controllable  lactors  Xj,  ....  under  investigation  and  that  a 
maximum  of  N  =  nm  computer  runs  are  available  for  use  by  the  "OPTIMIZER", 
where  m  iterations  are  to  be  made  at  each  point  (Xj,  ....  X^)  in  order  to  reduce 
statistic  al  variation  by  obtaining  average  observed  responseo. 

A.  RANDOM  SEARCH 

An  overall  search  region,  defined  by  user  input  of  upper  and  lower  bounds 
on  each  of  the  k  factors,  is  the  region  to  which  the  search  for  the  "optimum" 
is  restricted.  As  its  name  implies,  this  search  technique  selects  points  at 
random  from  within  the  overall  search  region.  Each  of  these  k-dimensional 
points  determines  the  values  of  the  controllable  factors  to  be  used  in  obtaining 
an  observed  response.  When  all  N  computer  runs  have  been  used,  the  point 
which  yielded  the  largest  observed  response  is  the  "optimum"  identified  by 
Random  Search. 

If  a  user  desires  to  use  the  Random  Search  option  of  the  "OPTIMIZER", 

’ne  must  input  the  values  of  k,  m,  and  n,  and  must  also  specify  the  upper  and 
lower  bounds  for  each  controllable  factor. 


Although  these  techniques  are  described  in  terms  of  a  search  for  a  maximum, 
they  are  equally  applicable  in  a  search  for  a  minimum. 
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B.  SINGLE-FACTOR  APPROACH 


The  Single-Factor  approach,  as  its  name  implies,  concentrates  on  one 
factor  at  a  time.  Specifically,  when  there  are  k  controllable  factors,  Xj,  .  .  . 

a  starting  point  (Cj . C^)  is  selected  by  the  user,  and  a  computer  run  made 

at  this  poinl.  A  step  size  Aj  is  chosen  and  the  Xj  value  is  changed  to  Cj  -  Aj 
with  all  the  other  X/s  remaining  at  their  starting  values.  A  computer  run  is 
made  at  this  new  point.  Similarly,  a  computer  run  is  made  at  a  point  where 
Xj  =  Cj  +  Aj  with  the  other  XBs  remaining  the  same.  If  the  observed  response 
at  either  of  those  two  points  is  better  than  at  the  starting  point,  the  search  is 
extended  in  the  direction  of  the  most  improvement,  and  with  all  the  other  factors 
held  constant,  a  new  computer  run  is  made  corresponding  to  a  value  of  Cj  ■  <JAj 
or  Cj  +  2Aj  for  Xj,  as  is  appropriate.  The  search  continues  in  the  same 
direction  as  long  as  improvement  continues  to  be  made. 

If  a  point  is  reached  at  which  improvement  is  no  longer  being  made,  the 
best  previous  point  is  regarded  as  a  new  starting  point,  a  step  size  A^  is  chosen, 
and  the  process  is  repeated,  changing  only  while  keeping  the  values  of  the 
other  factors  fixed.  After  factor  X^  is  examined,  the  cycle,. i.g  completed  by  then 
considering  Xj  again.  At  the  end  of  a  cycle,  each  A.  is  halved  before  a  new 
cycle  begins.  This  procedure  continues  with  each  factor,  in  turn,  until  either 
the  process  is  terminated  by  the  identification  of  an  "optimum"  point,  or  the 
number  of  computer  runs  is  exhausted. 

In  addition  to  the  values  of  k,  n,  and  m.  the  user  of  the  Single- Factor  option 

of  the  "OPTIMIZER"  must  input  the  initial  values  of  the  c out  tollable  factors, 

i  o.  ,  the  starting  point  (Cj,  .  ,  C^).  and  an  initial  step  size  A^  for  each  factor 

X  ,  i  -  1 .  ...  k 

1 

C  RSM 

Unlike  the  Single-Factor  approach  which  varies  only  one  factor  at  a  time, 

RSM  permits  the  controllable  factors  to  be  varied  simultaneously.  Thus, 
although  the  direction  of  search  movement  always  remains  parallel  to  the  Xj 
axes  in  a  Single  -  Factor  search,  the  RSM  search  allows  movement  in  any  directum 

*  Developed  by  Friedman  and  Savage  (19-47). 

‘"Developed  by  Box  and  Wilson  ( 1 11  9  l  ) 
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However,  data  input  for  a  RSM  option  is  the  same  as  that  for  the  Single-Factor 
option,  That  is,  the  user  must  input  the  values  of  k,  n,  and  m,  the  starting 
point  {Cj,  ....  C^)  and  a  step  size  for  each  factor  X^,  i  —  1 ,  .  .  .  ,  k. 

RSM  uses  an  experimental  design  in  a  local  region  about  the  starting  point 
to  determine  the  estimated  gradient  direction  known  as  a  path  of  steepest 
ascent.  Computer  runs  corresponding  to  points  on  this  path,  which  provides 
the  direction  of  predicted  optimum  response,  arc  made  until  there  is  no  im¬ 
provement  in  the  observed  response,  at  which  time  the  whole  process  is 
repeated  within  a  smaller  experimental  region.  At  a  later  stage  in  RSM, 
additional  experiments  based  on  second  order  designs  may,  if  desired,  be 
conducted  to  determine  the  approximate  curvature  of  the  response  surface  in 
the  vicinity  of  a  given  point.  However*,  because  of  the  large  number  of 'com¬ 
puter  runs  required,  this  step  will  rarely  be  used  in  dealing  with  practical 
simulation  proolems.  Thus,  only  the  first  phase*  VJ  RSM  (the  initial  experiment 
and  the  exploration  of  the  path  of  steepest  ascent/  have  been  included  in  the  proto¬ 
type  "OPTIMIZER". 

In  general,  the  calculations  required  by  RSM  are  simpler  if  made  in  terms 
of  transformed,  or  coded,  factors  Xj,  ...  ,  x^,  where  xi  -  (X.  -  C^)/A.  for 

l  r  1 . k.  In  terms  of  the  coded  factors,  the  starting  point  of  the  search 

is  at  (0,  ....  0)  and  the  initial  step  size  is  unity  for  each  of  these  k  factors. 

In  the  following  discussion  it  will  often  be  more  useful  to  refer  to  the  coded 
factors  instead  of  the  original  factors. 

1 .  The  Initial  Experiment 

The  two  RSM  variations  which  are  included  in  the  prototype  "OPTIMI¬ 
ZER"  differ  only  in  the  initial  experiment  that  is  used  to  determine  the  path  of 
steepest  ascent,  Variation  I  of  RSM  uses  a  2  fractorial  or  a  l  *  fractional 
factorial^  of  minimal  size  as  the  initial  experiment,  while  Variation  II  uses 
a  simplex  design'*  involving  k  +  1  computer  runs,  the  minimum  required  for 
calculation  of  a  path  of  steepest  ascent. 


'for  a  discussion  of  second  order  designs,  see  Cochran  and  Cox  (J957). 
“\Sce  Cochran  and  Cox  (1917)  or  Davies  (1907), 

'See  Box  (1952), 
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Only  two  values  of  each  factor  X.  are  used  in  the  initial  experiment 
of  Variation  I.  These  values  define  a  local  region  about  the  starting  point 
(C|,  ...  i  C^)  and  consist  of  an  upper  value  U.  ~  C.  <  A.  and  a  lower  value 
L.  =  C.  -  A^.  The  experiment  is  constructed  under  symmetry  constraints 
which  guarantee  that  each  of  the  upper  and  lower  values  of  Xi  will  be  used  an 
equal  number  of  times.  In  addition,  each  of  these  two  values  of  X.  appears 
the  same  number  of  times  with  the  upper  and  lower  values  of  each  other  factor. 
For  example,  the  computer  runs  dictated  by  the  initial  experiment  for  the  case 
k  =  3  would  be: 


(Lj, 

L2* 

U3> 

r  <C1 

-Al- 

C2 

A2’ 

C3 

+  A3> 

(ur 

L2’ 

l3) 

=  (C1 

+  Aj, 

C2 

rvj 

<J 

i 

C3 

"  A3^ 

(I^l  i 

U2‘ 

l3) 

-  (c, 

-Al> 

C2 

+  A2- 

C3 

'  A3^ 

(ur 

U2' 

U3> 

-  (C1 

+  A  i  ’ 

+  a2, 

°3 

+  A3} 

In  terms  of  the  coded  factors,  the  initial  experiment  would  be: 

(-1.  -1.  +1) 

(+1.  -1,  -1) 

(-1.  +1.  -1) 

(+1.  +1.  +1) 

It  can  easily  be  seen  in  this  case  that  the  symmetry  conditions  hold. 

This  symmetry,  inherent  in  the  experiments  used  in  Variation  I, 
provides  efficient  estimation  of  gradient  direction  by  permitting  the  effect  of 
a  given  factor  X^,  to  be  estimated  by  averaging  over  a  number  of  combinations 
of  the  other  factors.  Thus,  these  experiments  provide  for  internal  replication 
which,  in  general,  allows  for  accurate  estimation  by  reducing  the  size  of  the 
statistical  error  associated  with  the  required  estimates.  The  major  drawback 
to  the  experiments  used  in  Variation  I  is  that  they  require  a  number  of  computer 
runs  which  is  a  power  of  two.  Because  a  minimum  of  k  +  1  computer  runs  is 
required  to  estimate  the  path  of  steepest  ascent  when  k  factors  are  involved, 
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one  would,  in  general,  not  wish  to  spend  many  additional  runs  in  initial 
experimentation.  If  k  —  35,  for  example,  the  initial  experiment  in  RSM  - 

Variation  I  would  require  64  computer  runs,  or  Z 8  more  than  the  minimum 

.  .  1 

required. 

Unlike  Variation  I,  Variation  II  of  RSM  is  based  on  an  initial  experimen 
which  always  uses  the  required  minimum  number  of  runs  to  obtain  the  path  of 
steepest  ascent.  This  initial  experiment  results  from  the  s implex  design 
discussed  by  Box  (1952).  The  nomenclature  used  in  describing  the  design 
arises  from  the  fact  that,  viewed  geometrically,  the  design  is  formed  by  the 
vertices  of  a  k-dimensional  simplex, 

Although  the  simplex  design  can  be  constructed  to  provide  an  estimate 
of  gradient  direction  which  is  as  accurate  as  that  provided  by  the  experiment 
of  Variation  I,  this  construction  is  extrerrfely  dependent  upon  the  assumption 
that  the  roeponse  surface  can  be  approximated  to  a  good  degree  by  a  hyperplane, 
i.e.  ,  a  linear  function,  in  a  local  region  about  the  starting  point.  Thus,  implicit 
in  Variation  I  is  the  assumption  of  an  approximating  hyperplane  within  the 
locality  around  the  starting  point  defined,  in  terms  of  the  coded  factors,  by 

-  +  1  for  i  =  1 . k.  A  similar  assumption  is  implicit  in  Variation  II, 

However,  for  Variation  II  to  provide  an  estimate  of  gradient  direction  with  the 
same  accuracy  as  the  initial  experiment  of  Variation  I,  it  turns  out  that  com¬ 
puter  runs  must  be  taken  corresponding  to  some  values  of  the  xVs  which  are 
quite  distent  from  the  interval  (-1,  +1).  For  example,  at  least  one  of  the 
x^s  must  assume  the  value  of  Vk  Thus,  for  k  -  100,  a  compulerjan  must  be 
taken  for  x.  -  10.  This  unhappy  circumstance  arises  from  the  orientation  of 
the  points  of  the  simplex  design  and  is  a  result  of  the.  ‘'vastness1’  of  hyperspace, 
which  was  rli si  u s so.!  by  Hooke  and  Jeeves  (1958)  In  actuality,  each  point  of  the 
simplex  design  and  each  point  of  the  fractional  factorial  are  exactly  the  satne 
distance  from  the  starting  point  in  k-dimensional  hyperspace.  However,  a 
point  in  the  fractional  factorial  with  coordinates  which  assume  the  values  of  •  1 
and  1  1  ui  nearer  the  starting  point  in  an  individual  coordinate  sense  than  is  a 
point  in  the  simplex  with  coordinates  which  may  assume  the  values  of  ■•-yj^'and 
yr  Although  none  of  these  coordinates  is  too  far  distant  from  the  interval 


Actually,  the  situation  is  not  quite  as  bad  as  it  might  seem.  This  point  is 
discussed  further  in  Section  IU.T3. 
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(-1,  i  1)  v  hen  k  is  relatively  small,  the  distance  becomes  intolerable  when  k 
is  large.  Therefore,  the  simplex  design  in  the  prototype  "OPTIMIZER"  was 
modified  to  insure  that  any  value  which  a  given  coded  factor  x^  must  assume  is 
not  farther  than"^Tfrom  the  starting  value  0.  Such  modification,  however, 
tends  to  result  in  a  less  accurate  estimate  of  gradient  direction. 

Ill  general,  then,  Variation  II  provides  an  initial  experiment  containing 
less  runs  than  the  initial  experiment  of  Variation  I,  while  Variation  I  estimates 
gradient  direction  more  accurately  than  does  Variation  II.  However,  once  the 
path  of  steepest  ascent  is  determined,  the  same  procedure  is  followed  by  both 
of  the  variations. 

2.  Following  the  Path  of  Steepest  Ascent 

In  addition  to  the  computer  runs  specified  by  the  initial  experiment,  a 
run  is  also  made  at  the  starting  point.  Based  on  the  observed  responses  obtained 
from  these  computer  runs,  the  "OPTIMIZER"  determines  the  path  of  steepest 
ascent.  Using  the  largest  observed  response,  the  "OPTIMIZER"  calculates  the 
point  on  the  path  which  provides  a  predicted  response  with  the  same  value,  and 
makes  a  computer  run  at  this  point.  (This  is  the  first  computer  run  taken  on 
the  calculated  path.  )  Another  computer  run  is  then  automatically  taken  on  the 
path-  a  distance  A  away  from  the  first  point  (in  the  coded  factors).  Because 
preliminary  studies  indicated  that  setting  A  -  ,  5  ^"provides  fairly  good  results/ 
this  is  the  value  of  A  in  the  "OPTIMIZER".  However,  if  the  user  should  decide 
that  another  value  for  A  would  be  more  appropriate,  he  has  the  option  of  in¬ 
putting  that  value. 

After  the  two  initial  computer  runs  are  taken  on  the  path  of  steepest 
ascent,  additional  runs  are  taken  on  the  path,  each  one  a  di  stance  A  away  from 
the  previous  run,  so  long  as  an  improved  response  is  observed.  If  a  point  is 


This  second  computer  run  on  the  path  is  taken  regardless  of  whether  or  not  the 
first  computer  run  provided  improvement.  This  procedure  guards  against  the 
possibility  that  th first  point  was  not  far  enough  out  on  the  path. 

2  It  should  be  noted  that  in  a  previous  study,  McArthur  (1961)  found  that  A  =  I.  2 
appeared  to  give  the  best  results  for  cases  involving  values  of  k  ranging  from 
2  to  10.  This  value  of  A  falls  into  the  interval  given  by  ,  5 yTT  for  k  =  2  and 
k  =  10,  i.  e,  ,  the  interval  (.707,  1.  5),  lending  some  credence  to  the  present 
choice  of  the  value  of  A, 


reached  at  which  the  observed  response  is  less  than  a  previous  observed 
response,  the  "OPTIMIZER"  uses  the  point  corresponding  to  the  maximum 
observed  response  as  the  center  of  a  new  experiment.  In  terms  of  the  original 

factors  Xj . X^,  the  center  point  has  moved  from  (C^ . C^)  to  a  new 

point.  When  this  occurs,  a  mw  step  size  for  each  factor  X^  is  defined  as  one* 
half  the  existing  step  size 

At  such  a  juncture,  the  "OPTIMIZER"  determines  whether  or  not  a 
sufficient  number  of  runs  remain  to  allow  all  the  runs  dictated  by  another 
experiment  to  be  carried  out.  If  so,  the  RSM  search  proceeds  in  a  manner 
similar  to  that  previously  described.  If  not,  the  "OPTIMIZER"  determines 
the  largest  number  of  factors  which  can  be  included  in  a  new  experiment  within 
the  remaining  number  of  runs  and  still  leave  a  minimum  of  two  runs  on  the 
path  of  steepest  ascent.  The  "OPTIMIZER"  uses  this  reduced  set  of  k#  (<k) 
factors  to  calculate  a  new  direction  of  steepest  ascent.  The  reduced  set  of 
factors  consists  of  those  k#  factors  corresponding  to  the  k*  factors  found  by 
the  previous  experiment  to  have  the  largest  effect  on  the  path  of  steepest  ascent. 

This  procedure  is  repeated  until  either  the  specified  number  of  com¬ 
puter  runs  are  exhausted  or  the  center  point  of  an  experiment  provides  a  better 
response  than  any  of  the  other  points  in  the  experiment.  When  either  of  these 
events  occurs,  the  best  observed  point  is  labeled  the  "optimum".  The  latter 
event  indicates  convexity  of  the  surface  in  the  vicinity  of  the  center  point, 
implying  that  application  of  a  second-order  design*  would  be  warranted.  The 
prototype  "OPTIMIZER",  however,  was  developed  under  the  assumption  that 
this  stage  in  the  search  process  would  rarely,  if  ever,  be  reached,  which 
follows  implicitly  from  the  assumption  of  an  extremely  limited  number  of 
available  computer  runs. 

D.  THE  accelerated  techniques 

The  accelerated  versions  of  Single -Factor ,  RSM -Variation  I,  and  RSM- 
Variation  II  operate  in  a  manner  similar  to  the  unaccelerated  versions. 

However,  when  movement  in  a  given  direction  (e,g,,  in  the  positive  X.  direction 


1 


See,  for  example,  Davies  (1967)  or  Cochran  and  Cox  (1957). 
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or  along  a  path  of  steepest  ascent)  provides  improvement  in  the  observed 
response  at  three  successive  points,  the  accelerated  versions  use  an  algorithm 
to  speed  up  movement  in  that  direction.  To  simplify  the  ensuing  discussion, 
only  RSM  acceleration  on  the  path  of  steepest  ascent  will  be  described.  The 
procedure  for  Single -Factor  acceleration  is  similar. 

If  the  assumption  of  an  approximating  hyperplane  is  reasonable,  it  can 
be  shown  *  that  an  estimate  y  of  the  value  of  the  response  at  the  coded  point 
(xj . x^.)  is  given  by 


y  s  b  +  £  b.x. 
'  o  ,11 


where  the  b.'s  are  estimates  obtained  from  the  experiment,  The  estimates 

(bj . b|J  determine  the  direction  of  the  path  of  steepest  ascent  in  terms 

of  the  coded  factors  and  thus,  any  point  on  the  path  may  be  denoted  by 
wbj,  ....  wb(.).  The  predicted  response  at  this  point  is  given  by 


b  +  £  b,  (wb  ) 

o  j  i  i 


=  b  +  w  2  b.  . 
°  1  ‘ 


Now,  as  described  in  the  previous  section,  each  point  on  the  path  is  a  distance 
of  A  from  the  previous  point.  Therefore,  if  the  initial  point  on  the  path 
corresponds  to  w  =  w^,  the  second  point  on  the  path  then  corresponds  to 

w  =  w,  =  w  +  (A f\I 2!  b.^).  In  general,  the  jltl  point  on  the  path  corresponds 

i  i  1 


to  W  s  W.  = 


W.  =  W  +  (j  -  1)  (A/  V  £  b,  ). 
J  1  i  1 


*See,  for  example,  Davies  (1967)  or  Cochran  and  Cox  (1957), 


Thus,  the  distance  between  the  point  corresponding  to  w.  and  the  point 


corresponding  to  1* 


£  (w.b.  -  w,  .  b, )* 

i  ;  »  j-1  ‘ 


K  ?  2 
£  b.  (w.  -  w.  .) 

I  1  J  J"1 


2  Va2/£s.2 


as  required.  By  means  of  some  algebraic  manipulation,  the  predicted  response 
fh 

at  the  j  point  on  the  path  may  be  written  as 


y,  =  jA  +  B 
J  o  o 


where 


Aq  =  A\/Xb.‘ 


B  =  b  +  (w . 
o  o  1 


k  2  ^  2 

£  b.  )  £  b. 


Thus,  the  predicted  response  is  a  linear  function  of  j. 

Again,  thij  result  is  contingent  on  the  assumption  of  an  approximating 
hyperplane.  However  reasonable  this  assumption  within  the  starting  locality, 
it  quickly  loseu  credibility  as  points  further  out  on  the  path  are  considered. 

A  bit  more  general,  and  hence  more  plausible,  assumption  might  be  that  the 
predicted  response  is  approximately  a  quadratic  function  of  j.  That  is,  the 
assumption  might  be  made  that 

=  +jBl  +  C1 
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Based  on  this  assumption,  the  observed  responses  from  the  first  three 
points  on  the  path  of  steepest  ascent  may  be  used  to  estimate  the  values  of 
A,,  B .  ,  and  C. .  In  addition,  the  estimated  location  of  an  extremum  can  be 

l  i  1 

-R1 

obtained  from  j  =  - ,  where 

2A, 


Aj  .5  (y3  -2y2  +Yl) 


i . 5y3-2, 5yt 


An  acceleration  algorithm,  based  on  the  preceding  discussion,  waa  developed 
for  optional  use  within  the  "OPTIMIZER".  Specifically,  this  algorithm  contains 
thd  following  five  steps: 

(1)  Observe  the  response  y^  at  the  initial  point  on  the  path 
and  the  response  y^  at  the  second  point  on  the  path. 

(Z)  If  Yz>  Yx<  observe  response  y3  at  the  third  point  on 
the  path. 

(3)  If  Y3>  Yz>  Yy  compute  the  coefficients  Aj  and  Bj  and 

r-Bn 

determine  j  =  I -  ,  that  is,  the  truncated  value  of  - . 

12A.J  2A. 


that  is,  the  truncated  value  of 


(4)  If  j  >  1 1  or  j  <  1 ,  set  j  =  1 1 .  Otherwise ,  retain  the 
calculated  value  of  j.  (A  vaLue  of  j  <  1  provides  as  an 
extremum  the  minimum  of  a  quadratic  function  which  is  concave 
upward,  reflecting  the  fact  that  the  observed  responses  are 
increasing  at  an  increasing  rate.  In  this  situation,  setting 
j  -  11  yields  a  point  which  1b  a  reasonable  distance  on  the 
path  in  view  of  the  fact  that  the  quadratic  assumption  may 
not  hold  and  the  observed  responses  may  contain  random  error. 
Similarly,  if  j  >  11,  setting  j  =  11  guards  agaii.st  venturing 
too  far  out  on  the  path.) 


-285- 


(5)  Observe  the  response  y corresponding  to  the  j  point.  If 

y^  >  y^,  repeat  the  preceding  steps  using  y.  as  the  starting 

point.  If  y-j  <  ,  backtrack  on  the  path  to  get  y^  j. 

If  yj_j  <  y^,  return  to  y^  and  continue  on  the  path  in  the 

standard  manner.  If,  however,  y,  ,  >  y.,  observe  y,  ,,  etc, 

J )  J*2 

as  long  as  improvement  continues,  and  when  improvement 
stops,  say  at  y^,  and  if  y^>y3,  perform  a  new  experiment 
with  the  h1'1  point  as  center.  If  y^  <  y^,  however,  go  hack 
to  y3  and  continue  on  the  path  in  the  standard  manner. 


Similar  steps  are  followed  for  the  accelerated  version  of  the  Single -Factor 
approach,  the  only  difference  being  that  only  one  factor  is  changed  at  a  time 
and  the  steps  are  made  in  terms  of  a  A.  change  for  factor  X.,  which  corresponds 
to  a  change  of  one  unit  in  the  coded  factor  x. . 
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III.  EMPIRICAL  INVESTIGATION  OF  THE  SEARCH  TECHNIQUES 

Because  no  analytical  basis  exists  for  evaluation  of  the  performance  of  the 
search  techniques  relative  to  characteristics  which  may  vary  from  simulation  to 
simulation,  it  is  necessary  to  perform  empirical  studies  in  order  to  obtain  a 
comparison  of  the  techniques.  Although  such  empirical  studies  could  be  based 
on  unknown  response  surfaces  generated  by  actual  computer  simulations,  the 
current  research  effort  used  response  surfaces  which  were  constructed  from 
known  mathematical  relationships  specifically  chosen  for  the  investigation.  This 
procedure  has  many  advantages  associated  with  it,  Response  surfaces  con¬ 
structed  in  this  manner  present  difficulties  of  the  same  type  as  difficulties 
encountered  in  a  full-scale  simulation.  Because  full-scale  simulations  are  not 
required  in  the  evaluation,  however,  there  is  a  substantial  saving  in  costs 
(associated  with  modeling  ,  programming,  and  computing) ,  Also,  the  known 
mathematical  relationships  generating  the  response  surface  can  be  so  con¬ 
structed  that  the  true  optimum  is  known,  thus  permitting  a  comparison  of  the 
"optimum"  found  by  a  search  technique  with  the  actual  optimum,  in  addition  to 
a  comparison  with  the  "optimum"  found  by  any  other  technique. 

Although  empirical  investigations  of  search  techniques  on  known  response 
surfaces  were  carried  out  by  Brooks  (1959)  and  McArthur  (1951),  they  con¬ 
sidered  cases  where  only  a  few  controllable  factors  were  involved.  (The 
majority  of  cases  considered  were  two-factor  problems.)  Although  Brooks  and 
McArthur  used  different  criteria  for  evaluating  the  performance  of  the  search 
techniques,  their  conclusions  agreed.  These  conclusions  were,  in  general,  to 
use  RSM  if  only  a  small  number  of  factors  were  involved,  and  to  use  Random 
Search  if  a  large  number  of  factors  were  involved.  Although  Brooks  did  not 
give  any  definite  value  for  distinguishing  between  a  small  number  and  a  large 
number  of  factors,  McArthur's  study  indicated  that  in  any  case  involving  more 
than  three  factors,  Random  Search  should  be  used. 

The  current  study  has  benefited  from  both  of  these  previous  investigations, 
particularly  from  McArthur's  work,  which  used  ten  properties  to  characterize 
optimization  search  problems,  A  similar  classification  method,  described  in 
the  following  sections,  was  used  in  the  current  effort. 


A.  CRITERIA  USED  IN  THE  INVESTIGATION 


As  a  first  step  in  planning  for  an  investigation  of  the  performance  of  the 
various  search  techniques,  the  properties  suggested  by  McArthur  were  revised 
to  serve  as  a  basis  for  categorizing  simulation  situations.  This  revision  resulted 
in  the  following  list  of  characteristics: 

1.  The  number  of  controllable  factors,  _  - 

2.  The  number  of  available  computer  runs, 

3.  The  presence  or  absence  of  local  optima. 

4.  The  size  of  the  random  error  (statistical  variation)  present. 

5.  The  distance  of  the  starting  point  of  the  search  from  the  true 

optimum  point. 

6.  The  relative  activity  of  the  factors. 

7.  The  presence  or  absence  of  interaction  among  the  factors, 

The  evaluation  of  search  technique  performance  involved  the  application  of  each 
of  the  techniques  on  response  surfaces  constructed  to  have  specific  combinations 
of  these  characteristics. 

In  order  to  normalize  the  measure  of  performance  of  the  techniques  on  the 
various  resultant  surfaces,  it  was  decided  to  use  the  concept  of  relative  gain 
based  on  true  responses  rather  than  on  observed  responses.  (A  true  response 
is  the  response  which  would  have  been  observed  if  no  random  error  were 
present.)  Specifically,  if 

(a)  Rf  denotes  the  true  response  at  the  "optimum"  found  by 
a  search  technique, 

(b)  Rg  denotes  the  true  response  at  the  starting  point  of  the 
search,  and 

(c)  Rq  denotes  the  true  optimum  response, 

then  gain  will  be  defined  as  (Rp  -Rg)/(RQ  'Rg).  That  is,  the  gain  is  a  fraction 
of  the  maximum  possible  improvement,  Rq  *  Rg,  that  could  be  made.  It  should 
be  noted  that  although  the  gain  can  never  be  greater  than  100  percent,  it  is 
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conceivable  that  it  may  be  negative  in  some  cases.  A  negative  gain  might  result, 
for  instance,  because  of  the  presence  of  random  error  which  resulted  in  a  search 
technique  identifying  an  "optimum"  point  which  actually  yields  a  true  response 
less  than  that  of  the  starting  point. 

In  light  of  this  definition  of  gain  and  the  various  simulation  characteristics 
mentioned  previously,  the  evaluation  of  search  technique  performance  attempted 
to  provide  at  least  a  partial  answer  to  one  main  question.  Specifically,  this 
question  is:  if  k  controllable  factors  are  under  consideration  and  N  computer 
runs  are  available,  what  is  the  best  search  technique  to  use  and  how  much  gain 
is  to  be  expected  from  using  this  technique  on  response  surfaces  having  various 
characteristics  ? 

The  specific  aspects  of  the  characteristics  chosen  for  inclusion  in  this  study 
are  delineated  in  the  following  sections. 

1 .  The  Number  of  Controllable  Factors 

The  number  of  input  variables  for  computer  simulations  of  military 
operations  research  problems  tends  to  be  extremely  large.  Although  only  a 
subset  of  these  variables  will  be  included  as  controllable  factors  in  any  particular 
optimization  problem,  the  number  of  factors  in  this  subset  can  easily  be  20,  120, 
or  even  larger.  Very  rarely  does  one  see,  in  the  simulation  framework,  an 
optimization  problem  involving  only  a  few  (e.g.,  less  than  ten)  controllable 
factors , 

To  permit  an  examination  of  the  performance  of  the  search  techniques 
in  situations  involving  a  reasonable  number  of  controllable  factors,  it  was 
decided  to  deal  with  30-factor  and  120-factor  problems.  These  values  provide  at 
least  rough  bounds  for  the  number  of  factors  a  simulation  user  is  willing  to  con¬ 
sider  in  a  trial -and -error  or  heuristic  search  for  an  optimum  solution.  Thus, 
in  the  current  study,  the  values  of  k  (the  number  of  controllable  factors)  con¬ 
sidered  were  k  -  30  and  k  =  120. 

This  choice  of  values  of  k  permits  the  use  of  fractional  factorials  which 
require  a  number  of  computer  runs  which  is  only  slightly  larger  than  the  value 
of  k.  For  example,  a  value  of  k  120  permits  the  use  of  a  fractional  factorial 
involving  128  computer  runs,  while  a  value  of  k  =  130,  for  instance,  would 
require  the  use  of  a  fractional  factorial  involving  256  computer  runs.  This 
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increase  of  128  required  computer  runs  corresponding  to  an  increase  of  only  10 
in  the  value  of  k  results  because  of  the  manner  in  which  the  fractional  factorials 
are  constructed*.  Implications  of  the  choice  of  the  values  of  k  for  inclusion  in 
this  study  are  discussed  in  Section  UI.D. 

The  Number  of  Available  Computer  Runs 

In  general,  the  simulation  user  knows,  at  least  roughly,  how  much  com¬ 
puter  time  he  may  expend  in  a  search  for  an  optimum  solution.  Of  course,  the 
user  usually  regards  this  amount  of  time  as  insufficient.  Nonetheless,  it  will 
implicitly  be  assumed  that  the  user  will  do  the  best  he  can- within  the  allotted 
time.  Because  the  running  times  of  various  simulations  are  inherently  different, 
to  say  nothing  of  the  differences  in  the  speed  of  different  digital  computers,  it  is 
more  convenient  to  deal  with  the  number  of  available  computer  runs,  H. 

In  the  typical  case,  the  available  number  of  computer  runs  is  relatively 
small  compared  with  the  number  of  controllable  factors.  In  view  of  this,  the 
cases  considered  in  the  present  study  involved  N  i  2k  computer  runs,  and  im¬ 
plicitly  assumed  that  only  one  iteration  would  be  made  at  a  given  point.  Specifi¬ 
cally,  the  performance  of  each  search  technique  was  evaluated  for  values  of 
N  =  .  5k,  1,1k,  1.5k,  and  2.0k,  In  other  words,  if  a  30 -factor  problem  were 
being  considered,  a  given  search  technique  would  locate  an  "optimum,"  first 
with  a  maximum  of  15  runs,  next  with  a  maximum  of  33  runs,  next  with  a  maxi¬ 
mum  of  45  runs,  and  finally,  with  a  maximum  of  60  runs. 

3.  Presence  or  Absence  of  Local  Optima 

The  response  surfaces  generated  by  computer  simulations  may  be  of 
many  different  varieties,  In  general,  a  simulation  user  searching  for  an  optimum 
hopes  he  is  facing  a  situation  in  which  there  is  a  global  optimum  but  no  local 
optima.  In  other  words,  he  is  hoping  for  a  unimodal  response  surface,  because 
if  he  does  find  an  optimum,  he  knows  that  it  is  the  global  optimum,  A  much 
worse  case  is  one  in  which  numerous  local  optima  exist  in  addition  to  the  global 
optimum.  In  this  situation,  the  user  is  facing  a  response  surface  which  consists 
of  various  peaks,  valleys  and  saddlepoints.  Hence,  if  he  does  locate  an  optimum, 


1 


See  the  discussion  of  the  situation  in  Section  II. C, 
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he  has  no  assurance  that  it  is  not  merely  a  local  optimum  which  provides  a 
response  much  worse  than  that  of  the  global  optimum.  Figure  3  illustrates  a 
unimod&l  response  surface  consisting  of  only  a  global  optimum  and  no  local 
optima,  while  Figure  4  illustrates  a  multimodal  response  surface  consisting  of  a 
global  optimum  and  many  local  optima.  Although  both  of  these  surfaces  are 
defined  by  only  two  factors  to  permit  illustration,  the  extension  to  a  larger 
number  of  factors  is  straightforward,  but  essentially  impossible  to  visualize. 
Because  there  are  many  ways  in  which  to  construct  unimodal  and  multimodal 
surfaces,  the  specific  mathematical  forms  giving  rise  to  the  surfaces  used  in 
the  study  will  be  briefly  described  here.  The  actual  construction  of  these 
surfaces  is  discussed  in  Section  III.B. 

The  unimodal  surface  which  possesses  only  a  global  optimum  was 
generated  by  a  quadratic  equation  in  the  k  factors.  The  equation  giving  rise  to 
this  type  of  response  surface  was,  in  generic  form, 

k  2 

y  =  -  X  A.  I  Zj  (Xj . Xk)]‘  - 

where  the  Zj's  are  linear  functions  of  the  Xj's.  The  specific  form  of  the 
functions  Z.  is  dependent  on  the  presence  or  absence  of  interaction  which  is 
discussed  in  Section  III.  A,  7,  and  the  A.  n  assume  the  values  0,  l,  or  4  as 
determined  by  the  relative  activity  of  the  factors,  *s  described  in  Section  III.  A.  6. 

The  multimodal  surface  having  local  optima  was  generated  by  a  com¬ 
bination  of  cosine  terms  damped  by  exponential  terms.  The  equation  used  in 
generating  the  response  surfaces  waB,  in  generic  form, 

y  =  -  E  Ai  [e"'69  |Zi  (X1 . |  cos  [  2it  Z.  (X. . XR)]  -  1  ] 

1 

As  in  the  previous  case,  the  Z^'s  are  linear  functions  of  the  X^'s,  dependent  on 
the  presence  or  absence  of  interaction.  In  addition,  the  values  of  the  A^'s  are 
determined  by  the  relative  activity  of  the  i»ctor&.  In  both  cases  the  generated 
response  surfaces  have  a  true  (global)  optimum  response  of  zero,  with  all  other 
true  responses  being  negative. 

Intuitively,  it  would  be  expected  that,  with  respect  to  Random  Search, 
the  adaptive  search  techniques  would  perform  less  well  on  a  multimodal  surface 
with  local  optima  (Figure  4)  than  on  a  unimodal  surface  with  only  a  global 
optimum  (Figure  3).  This  intuition  was  supported  by  the  data  which  was  observed 
in  the  empirical  study.  This  data  is  summarized  in  Section  III.C.4. 
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FIS.  3  SVIFICi  >ITI  M  L8UL  IPTIU 


4.  Size  of  Random  Error 


In  this  study,  two  different  sizes  of  random  error  have  been  considered, 
and  have  arbitrarily  been  labeled  large  and  small.  This  section  discusses  the 
choice  of  measurement  of  the  random  error. 

In  attempting  to  arrive  at  a  suitable  method  for  measuring  random 
error,  five  alternatives  were  considered.  These  were: 

(a)  Random  error  proportional  to  a  fixed  constant 

(b)  Random  error  proportional  to  the  true  optimum  response 

(c)  Random  error  proportional  to  the  true  response  at  the 
starting  point 

(d)  Random  error  proportional  to  the  difference  between  the 
true  optimum  response  and  the  true  response  at  the  starting 
point 

(e)  Random  error  proportional  to  the  difference  between  the  true 
optimum  response  and  the  true  response  at  some  fixed  distance 
from  the  true  optimum  point. 

In  defining  the  size  of  random  error,  it  seemed  reasonable  that  the  terms  "large 
error"  and  "small  error"  be  defined  consistently  so  that,  for  example,  small 
error  results  in  the  same  error  band  at  all  points  on  a  given  response  surface. 

Although  alternative  (a)  does  provide  a  consistent  definition  of  random 
error,  it  is  completely  independent  of  the  response  surface  itself.  If.  for 
example,  random  error  were  defined  as  arising  from  a  probability  distribution 
with  standard  deviation  10,  the  effect  would  be  entirely  different  on  a  surface 
having  responses  in  the  range  (0,  .  100)  than  it  would  be  on  a  surface  having 
responses  in  the  range  (0,  100). 

At  first  glance,  alternative  (b)  seems  to  overcome  this  situation.  However 
because  the  initial  selection  of  mathematical  equations  for  generating  response 
surfaces  resulted  in  true  optimum  responses  of  zero,  this  alternative  would 
result  in  random  error  which  was  identically  zero.  Although  the  mathematical 
equations  could  be  modified  by  the  addition  of  a  constant  term  to  provide  a  non¬ 
zero  optimum,  this  procedure  would  result  in  objections  similar  to  those  raised 
for  alternative  (a). 
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By  defining  random  error  proportional  to  the  true  response  at  the 
•tarting  point,  alternative  (c)  result!  in  inconaistenciea .  This  is  due  to  the 
fact  that,  in  general,  the  farther  the  starting  point  from  the  true  optimum,  the 
smaller  the  true  response  at  that  point.  Thus,  a  starting  point  relatively  close 
to  the  true  optimum  point  would  tend  to  result  in  a  smaller  error  band  than  would 
a  starting  point  relatively  far  from  the  true  optimum.  In  addition,  objections 
similar  to  those  raised  for  alternative  (b)  also  exist. 

If  random  error  is  defined  proportional  to  the  difference  between  the 
true  optimum  response  and  the  true  response  at  the  starting  point,  as  in  alterna¬ 
tive  (d),  a  certain  amount  of  stability  exists  in  that  the  addition  of  a  constant  to 
all  responses  on  the  surface  does  not  change  the  magnitude  of  the  error,  How¬ 
ever,  as  in  the  case  of  alternative  (c),  the  magnitude  of  the  random  error  is 
larger  for  starting  points  far  from  the  true  optimum  point  than  it  is  for  starting 
points  near  to  the  true  optimum  point. 

Under  alternative  (e)  the  size  of  random  error  would  be  consistently 
defined  on  any  given  response  surface  of  given  characteristics,  regardless  of 
the  location  of  the  starting  point.  In  addition,  this  measure  of  random  error  is 
stable  under  any  translation  of  the  surface  in  the  y-direction.  Thus,  this 
alternative  was  chosen  as  the  definition  of  random  error  to  be  used  in  the 
empirical  study.  Specifically,  random  error  was  defined  proportional  to  the 
difference  between  the  true  optimum  response  and  the  true  response  at  a. point 
a  distance  of  A  away  (in  terms  of  the  coded  factors)  on  the  line  joining  the 
starting  point  and  the  true  optimum  point.  If  this  difference  is  denoted  by  D, 
small  random  error  was  generated  from  a  Normal  distribution  with  mean  zero 
and  standard  deviation  0.  ID,  while  large  random  error  was  generated  from  a 
Normal  distribution  with  mean  zero  ahd  standard  deviation  1.0D. 

From  a  practical  standpoint,  the  main  disadvantage  to  this  definition 
of  random  error  is  that  it  may  be  somewhat  difficult  for  a  simulation  user  to 
extrapolate  it  to  a  simulation  context. 

5.  Distance  of  Starting  Point  from  True  Optimum  Point 

If,  in  terms  of  the  original  factors,  the  true  optimum  point  is  denoted 

by  (Tj . T^),  one  aim  of  the  simulation  user  is  to  provide  a  starting  point 

(Cj . Ck)  that  is  not  extremely  distant.  To  investigate  the  effect  of  the 

distance  of  starting  point  from  true  optimum  on  search  technique  performance, 
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some  definition  of  "near"  and  "far"  distance  must  be  made.  Actual  distance  in 
the  factor  space  does  not  provide  a  suitable  measure  because  different  values 
assigned  to  a  step  size  A^  would  result  in  the  distance  being  traversed  in  differ¬ 
ent  numbers  of  steps.  It  is  reasonable  to  take  this  into  account  in  order  to  pro¬ 
vide  a  stable  measure. 

The  user's  specification  of  the  C/s  and  the  A/s  result  in  an  initial  search 
region  in  the  locality  of  the  starting  point.  If  the  true  optimum  point  is  near,  there 
should  be  a  fairly  good  chance  that  C.^  -A.  <  T,  <  +  A^  for  any  i.  The  chances 

diminish  as  the  true  optimum  point  becomes  more  distant.  Assume  that  the  user 
chooses  the  C's  and  the  A^'s  in  such  a  manner  that  Prob  (Ci  -A^  s  T.  $  C.  +  A^)  is 
equal  for  each  i.  It  is  not  unrealistic  to  also  assume  that  each  C.  selected  by 
the  user  is  approximately  a  Normal  random  variable  with  expected  value  T.  and 
standaid  deviation  which  is  a  function  of  A^.  Definitions  of  "near"  and  "far"  dis  - 
tance  were  baaed  on  this  framework.  Specifically,  a  "near"  distance  was  defined 
to  exist  when  Prob  (Cj  -A^  <  <  C^+A^)  -  80%.  Similarly,  a  "far"  distance  was 

defined  to  exist  when  Prob  (C^  -  A^  <  +  A^)  s  20%.  It  follows  that  for  near 

distance,  Cj  is  Normally  distributed  with  mean  and  standard  deviation  0.78A. , 
while  for  far  distance,  is  Normally  distributed  with  mean  and  standard 
deviation  3.  95  A. . 


6.  Relative  Activity  of  the  Controllable  Factors 

Although  a  number  of  controllable  factors  may  be  thought  to  affect  the 
response  generated  by  a  computer  simulation,  in  actual  practice  it  is  usually 
found  that  some  of  these  factors  have  negligible  effect  on  the  response.  In  other 
words,  the  response  is  insensitive  to  changes  in  these  factors.  Such  factors 
will  be  labeled  inactive,  and  the  remaining  factors  labeled  active. 

It  is  quite  conceivable  that  differences  in  the  relative  activity  of  the 
controllable  factors  may  dictate  different  search  techniques.  Thus,  two  levels 
of  activity,  low  and  high,  were  considered.  Low  activity  was  defined  to  exist 
when  only  20%  of  the  controllable  factors  actually  generated  the  response  sur  - 
face.  High  activity  was  defined  to  exist  when  80%  of  the  controllable  factors 
generated  the  response  surface. 


7.  Presence  or  Absence  of  Interaction 


Interaction  is  said  to  exist  when  the  equation  defining  the  response 

surface  contains  terms  involving  products  of  the  factors.  Thus,  interaction 

is  absent  when  no  term  in  the  defining  equation  involves  products  of  the  factors. 

When  interaction  is  present,  the  optimum  value  of  a  given  factor  depends  upon 

the  specific  values  of  other  factors.  When  interaction  is  absent,  the  optimum 

value  of  a  given  factor  is  independent  of  the  specific  values  other  factors  may 

2  2 

assume.  For  example,  if  a  response  surface  were  defined  as  y  a  10  -  Xj  - 
and  a  maximum  were  desired,  =  0  would  provide  the  optimum  response  re¬ 
gardless  of  the  value  of  X,.  However,  if  the  surface  were  defined  as 
2  >  c 

y=  10  -  Xj  -2XjX2  -X2,  the  value  of  Xj  producing  the  optimum  response  would 
depend  upon  .  the  value  of  X?.  It  can  easily  be  shown,  of  course,  that  this 
value  of  Xj  would  be  given  by  Xj  =  -X^ . 

In  Section  III. A.  1,  which  deals  with  the  presence  or  absence  of  local 
optima  on  the  response  surfaces,  the  mathematical  forms  generating  the  surfaces 

involved  functions  of  the  type  Z ^  (Xj . X^).  For  sufaces  without  interaction, 

this  function,  in  essence,  was  defined  as  Z.  (X^ . X^)  =  (X^  -Cj)/Aj.  For 

surfaces  with  interaction,  (Xj,  . . .  ,  X^)  was  determined  by  the  application  of 
a  Helmert  transformation  to  the  active  factors.  This  transformation  produces 
various  two -factor  products  in  the  equation  for  the  response  surface.  Further 
details  are  provided  in  the  following  section. 

B.  CONSTRUCTION  OF  THE  RESPONSE  SURFACES 

The  actual  construction  of  the  response  surfaces  used  in  the  empirical 
studies  was  accomplished  by  a  set  of  computer  programs  and  subprograms 
designed  to  provide  a  response  surface  according  to  the  desired  characteristics 
(as  described  in  the  previous  sections),  to  apply  each  of  the  search  techniques 
on  the  surface,  to  record  the  gains  obtained  by  the  techniques,  and  to  repeat 
these  steps  for  a  number  of  iterations  on  each  type  of  constructed  surface.  The 
use  of  a  number  of  iterations  permits  a  fairly  accurate  estimate  of  the  expected 
gain  provided  by  the  use  of  a  given  search  technique  on  a  particular  type  of 
response  surface. 

Before  a  description  is  given  of  the  process  involved  in  the  construction  of 

a  response  surface,  it  should  be  mentioned  that  all  calculations  involved  in  the 

construction  are  based  on  the  coded  factors  x.  =  (X.  -  C,)/A..  As  stated 

i  i  i  i 
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previously,  this  transformation  results  in  an  optimization  problem  involving 

factors  (xj . x^),  a  starting  point  <0 . 0),  and  a  step  size  of  one  unit 

fot  each  factor.  The  remainder  of  the  discussion  will  refer  to  tkxe  transformed 
factors  (xj,  ....  x^),  unless  the  original  factors  (Xj,  ....  Xfc)  are  specifically 
identified. 

The  process  of  response  surface  construction  begins  with  the  selection  of 
the  point  which  produces  the  true  optimum.  If  the  true  optimum  point  is  denoted, 

in  terms  of  the  original  factors,  by  (Tj . Tk),  then,  in  the  transformed 

factor  space,  the  true  optimum  point  has  the  coordinates  given  by  tj  =  (T.  -C^)/^, 
for  i  -v  1 . k.  It  directly  follows  from  the  discussion  of  the  distance  cri¬ 

terion1  that  the  point  (tj,  , .  . ,  t^)  can  be  determined  by  generating  each  tj  from 
a  Normal  distribution  with  mean  0  and  standard  deviation  which  is  equal  to  3.95 
fov  "far"  distance  and  0.78  for  "near"  distance.  For  each  iteration,  this  is  the 
manner  in  which  the  true  optimum  point  i»  selected. 

After  this  point  is  selected,  the  values  of  the  A^  coefficients  used  in  the 
equation  of  the  response  surface  (Section  III.  A.  3)  are  determined.  The  activity 
of  the  factors  determines  the  values  of  A. 'a  and  the  specific  factors,  x.,  which 
are  to  define  the  response  surface.  In  the  computer  program,  80%  of  the  factors 
are  selected  as  active  factors  in  the  case  of  "high"  activity  and  20%  are  selected 
as  active  in  the  case  of  "low"  activity.  Of  the  corresponding  coefficients,  half2 
are  set  equal  to  1 . 0  and  the  other  half  are  set  equal  to  4.0.  In  each  step  of  this 
process,  the  choice  of  the  active  x^s  and  the  selection  of  the  particular  values 
to  be  given  to  the  A.'s  are  made  randomly. 

At  this  juncture,  then,  initial  preparation  for  the  construction  of  the  surfaces 
is  completed,  and  there  exists  a  subset  of  k*  controllable  factors  which  are  active 
and  which  directly  affect  the  response.  For  convenience  in  the  following  discus¬ 
sion,  it  will  be  assumed  that  this  subset  consists  of  the  t’irst  k#  of  the  k  factors, 
i.e.,  (xj,  ..  .  ,  xk#).  Of  course,  the  actual  k*  active  factors  would  probably 
never  be  the  first  k*  factors,  but  having  been  randomly  chosen,  would  most 


*See  Section  III.  A.  5. 

2It  should  be  noted  that  if  all  the  A^'s  were  given  the  same  value,  circular  con¬ 
tours  would  result.  The  way  they  are  defined  gives  rise  to  more  realistic 
elliptical  contours. 
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likely  be  interspersed  within  the  set  of  k  factors.  The  assumption,  however, 
prevents  the  mathematical  notation  from  becoming  horrendous. 

The  required  mathematical  functibns  Z.  (Section  111. A.  3)  are  obtained  from 
equivalent  functions  z.  of  the  transformed  factors.  The  z^,  which  are  defined  in 
terms  of  the  active  subset  of  the  controllable  factors,  are  dependent  upon  the 
presence  or  absence  of'interaction.  If  interaction  is  to  be  absent,  each  z^  1b 
defined  simply  as  a  x.  -  t.  which,  in  terms  of  the  original  factors,  is  equiva¬ 
lent  to  Z.  a  (X.  -  T. ) /A. .  Thus,  the  resulting  mathematical  equation  is  one 
which  does  not  contain  any  terms  involving  more  than  one  factor.  Viewed  geome 
trically,  the  response  surface  which  is  produced  in  this  situation  has  its  axes 
parallel  to  the  coordinate  axes.  If  interaction  is  to  be  present,  a  Helmert 
transformation1  is  used  to  produce  the  interaction.  This  transformation  is 
given  by: 


1 


k* 


_  r  x. 

Vk*  1  1 


z. 

J 


j-1 

?  x‘ 

m 


for  2  <  j  <  k» 


The  resulting  equation  defining  the  response  surface  contains  interaction  in  the 
form  of  terms  which  involve  two -factor  products.  Because  of  these  product 
terms,  the  axes  of  the  response  surface  are  not  parallel  to  the  coordinate  axes. 

After  the  z  's  have  been  defined,  the  appropriate  choice  of  surface  type  is 
made,  dependent  upon  the  presence  or  absence  of  local  optima.  In  the  absence 
of  local  optima,  the  response  surface  is  generated  by 

k*  2 

y a  -  f  . *k*>] 

which,  regardless  of  whether  or  not  interaction  is  present,  is  a  negative  definite 
quadratic  equation  in  (x^,  ....  x^),  having  a  maximum  value  of  y  =■  0.  As 
written,  the  equation  is  a  canonical  representation. 

^See,  for  example,  Kendall  ar.d  Stuart  (1963)  for  a  discussion  of  this 
transformation. 
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If  local  optima  are  present,  the  response  surface  is  generated  by 


y  = 


k* 

-r 

1 


[e-*69  |zi  (xl*  xk*> 


cos  [  2n  zi  (x.,  . . .  , 


which  also  has  a  maximum  value  of  y  -  0.  This  response  surface  has  many 
local  maxima,  local  minima,  and  saddle  points,  as  may  be  seen  from  Figure  4, 
which  is  a  surface  defined  by  this  equation  for  the  case  where  only  two  active 
controllable  factors  are  involved  (i.e.,  k*  -  2). 


For  any  particular  computer  run  involving  the  active  controllable  factors 
(xj,  ....  x^),  the  response  obtained  is  calculated  from  the  appropriate  equation 
given  above.  However,  the  remaining  characteristic,  random  error,  must  still 
be  considered.  Thus,  after  a  true  response  has  been  calculated,  random  error 
is  generated  from  the  appropriate  Normal  distribution*  and  Is  added  to  this 
response. 

The  response  surfaces  constructed  by  these  procedures  were  used  to 
evaluate  search  technique  performance. 


C  .  E  VALUATION  OF  SEARCH  TECHNIQUE  PERFORMANCE 

To  evaluate  the  performance  of  the  search  technique  on  response  surfaces 
c£  differing  characteristics,  each  search  technique  must  be  applied  to  each  of 
the  various  response  surfaces.  Furthermore,  to  provide  a  reasonably  accurate 
assessment  of  performance  (measured  in  terms  of  relative  gain),  a  number  of 
iterations  must  be  made  to  dampen  out  random  fluctuations  arising  from  the 
statistical  variation  present  in  the  procedures  used  to  construct  response  sur  ¬ 
faces  of  a  given  type,  and  from  the  random  error  with  which  the  true  responses 
are  contaminated.  For  the  relatively  small  number  of  characteristics  investi¬ 
gated  in  this  study,  the  number  of  actual  surface  searches  conducted  and,  there¬ 
fore,  the  number  of  computer  runs  used  were  quite  large. 

A  total  of  64  different  types  of  response  surfaces  were  generated,  corre¬ 
sponding  to  all  of  the  combinations  of  various  characteristics  considered.  These 
combinations  arise  from: 


V 


See  Section  III.  A.  4. 
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.Number  of  controllable  factor*: 
Local  Optima:  . 

Random  Error; 

Distance: 

Activity: 

Interaction; 


30  or  120 

Presence  or  Absence 
Large  or  Small 
Far  or  Near 
High  or  Low 
Presenca  or  Absence 


Each  of  the  seven  search  techniques  in  the  prototype  "OPTIMIZER"  was 
applied  to  each  type  of  response  surface  and,  in  addition,  each  was  applied 
for  four  different  values  of  N,  the  maximum  number  of  available  computer  runs. 
This  implies  a  minimum  total  of  7  x  64  x  4  =  1792  searches  which  were  con¬ 
ducted,  In  addition,  however,  36  iterations  were  made  for  each  of  these 
searches,  which  means  that  a  grand  total  of  64,  512  searches  were  made  in  this 
investigation  1 


1 .  Definition  of  the  Overall  Search  Region 

Because  the  starting  point  of  (0,  ....  0)  and  the  initial  step  sizes  of 
unity  remained  constant  for  each  search,  the  actual  application  of  the  search 
techniques  could,  except  in  the  case  of  Random  Search,  proceed  directly  without 
requiring  any  additional  data  or  computation.  Random  Search,  however,  required 
the  assignment  of  an  overall  search  region.  Because  the  size  of  the  overall 
region  directly  affects  the  performance  of  Random  Search,  thiB  assignment 
posed  a  major  problem. 

Since  it  was  thought  that  Random  Search  would  make  a  relatively  poor 
showing  on  the  unimodal  surfaces,  it  was  decided  to  lean  over  backward,  so  to 
speak,  to  define  an  overall  region  which  provided  an  opportunity  for  Random 
Search  to  perform  well.  To  this  end,  the  overall  search  region  was  constructed 
in  such  a  manner  that  it  always  contained  the  true  optimum  point  within  a  rela¬ 
tively  small  volume.  Specifically,  if  t^,  the  jth  coordinate  of  the  optimum  point, 
fill  in  the  interval  (-.75,  .75)  the  overall  region  for  x^  was  defined  as  (-1,  l). 

If  t.  >  .75,  the  overall  region  for  x^  was  defined  as  (-1,  t.  +  .25),  and  if 

t.  <  -.75,  the  overall  region  for  x.  was  defined  as  (t.  -  .25,  1). 

J  *  s  J  j 


*The  computer  time  required  for  conducting  this  empirical  investigation  was 
provided  by  the  U.S.  Army  Strategy  and  Tactics  Analysis  Group  (STAG), 
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It  was  reasoned  that  if  Random  Search  made  a  poor  showing  under  these 
somewhat  ideal  conditions,  this  would  build  a  strong  case  for  jettisoning  the 
technique,  Such  a  poor  showing  was  anticipated,  but  did  not  materialize,  * 

2.  Procedure 

For  each  combination  of  characteristics,  the  applicable  computer 
routines  generated  the  response  surface  corresponding  to  the!  characteristics, 
calculated  the  true  response  at  the  starting  point,  applied  each  of  the  seven 
search  techniques  to  the  surface,  recorded  the  gain  achieved  by  each  technique 
for  each  of  the  four  values  of  N,  and  then  repeated  these  steps  until  a  total  of 
36  iterations  were  completed.  In  essence,  the  overall  investigation  could  be 
regarded  as  being  based  on  a  2^  x  4  x  7  factorial  experiment  with  36  observa¬ 
tions  per  cell. 

Different  sequences  of  pseudo -random  numbers  were  used  for  gen¬ 
erating  different  response  surfaces  within  a  given  type  of  surface  defined  by  a 
certain  combination  of  characteristics  .  Although  a  given  type  of  surface 
would  be  characterized  by,  say,  "near"  distance  and  "low"  activity,  surfaces 
of  this  type,  in  general,  would  have  starting  points  with  different  coordinates 
(although  still  conforming  to  the  "near"  distance  criterion)  and  would  consist  of 
different  subsets  of  active  factors  (although  the  same  precentage  of  factors 
would  be  active).  Different  sequences  of  pseudo-random  numbers  were  also 
used  in  the  generation  of  the  random  error  terms  which  were  added  to  the  true 
responses  to  yield  the  observed  responses. 

3.  Preliminary  Data  Analysis 

The  procedure  outlined  above  provides  for  the  statistical  independence 
of  the  responses  obtained  from  the  iterations  of  a  given  search  technique  on  a 
specific  type  of  surface,  as  well  as  the  independence  of  those  obtained  from 
different  types  of  surfaces.  However,  because  each  of  the  seven  search  tech¬ 
niques  was  applied  to  every  surface  of  a  given  type,  the  responses  obtained  by 


^See  the  discussion  in  Section  III.D, 

^In  this  discussion  a  surface  type  implies  a  surface  determined  by  a  set  of 
characteristics.  Surfaces  corresponding  to  different  sets  of  characteristics 
are  thus  of  different  types. 
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the  different  search  techniques  on  a  specific  type  of  surface  are  correlated,  The 
resulting  lack  of  statistical  independence  in  the  overall  experiment  creates  diffi¬ 
culty  in  an  analysis  of  the  data,  because  statistical  independence  is  an  assumption 
on  which  most  standard  methods  of  statistical  analysis  are  based.  However, 
certain  advantages  offset  this  disadvantage.  One  major  advantage  is  that  by 
using  iterations  in  which  the  search  techniques  were  applied  to  the  same  response 
surfaces  within  a  type,  differences  between  the  observed  responses  for  each 
technique  were  not  contaminated  by  any  effect  due  to  differences  in  the  individual 
response  surfaces*.  Another  advantage  to  using  the  same  surfaces  for  Iterations 
of  each  of  the  seven  search  techniques  is  that  computer  time  is  saved  by  having 
to  generate  only  36  surfaces  of  a  certain  type  instead  of  36  x  7  =  252  surfaces. 

Despite  the  lack  of  statistical  independence,  the  manner  in  which  the 
experiment  was  conducted  still  permits  an  analysis  of  variance  (ANOVA)  to  be 
performed,  with  such  an  analysis  based  on  the  assumption  of  nesting2  or  re¬ 
peated  measure**,  However,  ANOVA  seems  inappropriate  for  analyzing  the 
experimental  data  because  it  does  not  provide  a  direct  answer  to  the  previously 
asked  question  about  the  best  technique  to  use  on  a  surface  of  given  character¬ 
istics  and  the  gain  to  be  expected  from  using  this  technique.  In  essence,  a  more 
desirable  outcome  of  the  empirical  atudles  is  a  set  of  payoff  matrices  on  which 
to  base  the  selection  of  a  search  technique. 

A  preliminary  data  analysis  was  adopted  in  an  attempt  to  reduce  the 
accumulated  data  by  determining  which  of  the  response  surface  characteristics 
had  an  effect  on  the  gains  provided  by  a  given  search  technique,  This  prelimi¬ 
nary  data  analysis  involved  an  ANOVA  for  each  search  technique  and  combination 
of  k  and  N.  A  standard  ANOVA  based  on  the  assumption  of  a  completely  random¬ 
ized  factorial  experiment  could  be  performed  in  each  case  because  of  the  inde¬ 
pendence  inherent  in  each  combination,  As  one  would"expect,  correlation  did, 
however,  exist  between  various  of  the  ANOVA.  Nonetheless,  each  ANOVA 
strongly  indicated  that  the  gains  provided  by  the  search  techniques  were  quite 

*A  discussion  of  the  pros  and  cons  of  a  situation  like  this  is  given  in  Hillier  and 
Lieberman  (1967). 

2See,  for  example,  Schefftf  (1959). 

*See,  for  example,  Winer  (1962). 
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sensitive  to  *11  the  characteristics  considered,  except  for  the  presence  or 
absence  of  Interaction,  This  information  indicates  that  all  the  characteristics 
(except  interaction)  must  be  considered  in  the  construction  of  any  payoff  matrix. 

4.  The  Payoff  Matrices 

The  simulation  user  who  wishes  to  use  a  search  technique  in  an 
attempt  to  locate  an  optimum  solution  essentially  finds  himself  in  a  decision 
theory  framework*,  playing  a  game  against  Nature.  In  decision  theory  termino¬ 
logy,  the  possible  states  of  Nature  correspond  to  the  particular  combinations  of 
characteristics  possessed  by  the  response  surface  which  is  generated  by  the 
simulation.  The  user  has  a  set  of  possible  actions  corresponding  to  each  of 
the  seven  search  techniques.  For  each  action-state  of  Nature  combination 
there  is  a  payoff,  which  in  this  situation  may  be  assumed  to  be  expected  gain. 

Before  the  user  can  make  any  rational  decision  as  to  the  choice  of  an 
action  (i.e.  ,  a  search  technique),  however,  the  value  of  the  various  payoffs 
(i.e.  ,  expected  gains)  must  be  specified.  Although  the  true  payoffs  are  unknown, 
the  empirical  studies  have  provided  relatively  good  estimates  of  these  payoffs, 
Figures  5  through  12  contain  the  payofi  matrices  based  on  the  estimated  expected 
gains.  The  standard  error  of  each  estimated  expected  gain  is  given  to  indicate 
the  accuracy  of  the  estimation. 

If  a  user  were  faced  with  a  simulation  involving  30  controllable  factors 
and  had  available  a  maximum  of  45  computer  runs,  he  could  base  his  selection 
of  a  search  technique  on  the  data  contained  in  the  payoff  matrix  given  in  Figure  7. 
For  example,  suppose  the  state  of  Nature  was  known  to  produce  a  response 
surface  with: 

(a)  no  local  optima 

(b)  a  true  optimum  far  from  the  starting  point 

(c)  large  random  error 

(d)  high  factor  activity. 


*For  a  discussion  of  decision  theory  see  Chernoff  and  Moses  (1959)  or  Luce  and 
Raiffa  (1957),  for  example. 
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In  this  situation,  the  user  could  determine  from  Figure  7  that  the  search  tech* 
nique  having  the  largest  estimated  expected  gain  (.910)  was  RSM -Variation  I. 
Thus,  a  rational  selection  would  be  this  search  technique. 

If,  however,  the  true  state  of  Nature  is  unknown,  the  user  is  faced 
with  the  quandry  of  how  to  exploit  the  data  in  the  payoff  matrix  to  make  a  good 
selection  of  a  search  technique.  One  possible  solution  is  lor  him  to  adopt  a 
maximin  criterion1  whereby  he  would  select  a  search  technique  in  such  a  way 
as  to  maximize  the  minimum  gain  which  could  be  obtained.  In  general,  this 
criterion  results  in  what  is  known  as  a  mixed  strategy,2  A  mixed  strategy 
defines  a  probability  distribution  over  the  set  of  search  techniques,  and  the 
actual  search  technique  to  be  used  is  based  on  a  random  selection  governed  by 
the  probability  distribution , 

Application  of  the  maximin  criterion  provides  an  overly  conservative 
selection  of  search  technique,  however.  Essentially,  this  criterion  is  based  on 
the  pessimistic  assumption  that  Nature  is  a  malicious  opponent  who  will  select 
a  Btate  in  a  conscious  attempt  to  keep  the  simulation  user's  gain  se  low  as 
possible.  Since  this  pessimistic  assumption  is  usually  unwarranted ,  only  in 
extremely  rare  instances  would  the  user  base  his  selection  of  a  search  technique 
on  the  maximin  criterion. 

Although  the  true  state  of  Nature  is  usually,  if  not  always,  unknown  to 
the  user,  he  is  generally  not  in  complete  ignorance  because  he  tends  to  have 
some  information  about  the  simulation  itself.  Thus,  the  user  is  facing  a  condi¬ 
tion  of  partial,  rather  than  total,  ignorance  about  the  true  state  of  Nature,  Often 
he  can  summarize  this  partial  ignorance  by  means  of  a  probability  distribution 
over  the  possible  states  of  Nature.  The  existing  information  on  which  this 
probability  distribution  is  based  is  often  referred  to  as  prior  knowledge,  and  the 
resulting  probabilities  are  known  as  prior  probabilities.  In  those  cases  where 
no  relative  frequency  basis  exists  for  defining  a  probability  distribution,  "sub  ¬ 
jective"  probabilities  or  "degrees  of  belief"  may  be  used.  Although  much 


Because  the  entries  in  most  payoff  matrices  are  in  terms  of  losses  rather 
than  gains,  it  is  more  or  less  standard  to  think  of  a  minimax  criterion.  In 
the  present  situation  where  gains  are  being  considered,  a  maximin  criterion, 
the  mirror -image  of  a  minimax  criterion,  is  appropriate. 

•> 

‘‘See,  for  example,  Luce  and  Raiffa  (1957). 
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controversy*  exists  as  to  the  reasonableness  and  efficacy  of  subjective  probabi¬ 
lities,  It  will  be  tacitly  assumed  that  where  no  relative  frequency  based  probabi¬ 
lities  are  available,  the  simulation  user  can  specify  the  required  probabilities 
on  a  subjective  basis . 

It  is  reasonable  for  the  user  to  select  that  search  technique  which 

2 

provides  the  maximum  expected  gain  based  on  the  prior  probabilities  , 

Let  the  I***  state  of  Nature  be  denoted  by  S^,  let  the  prior  probability  that 

is  the  true  state  of  Nature  be  denoted  by  p.  and  let  the  estimated  expected  gain 

tK  3* 

corresponding  to  the  j  search  technique  when  is  the  true  state  of  Nature 
be  denoted  by  G^.  Then  the  search  technique  to  be  selected  would  be  the 
technique  which  provided  the  largest  value  of  £  p.  C.. .  As  a  simple  example, 

suppose  that  in  a  simulation  situation  with  k  =  30  and  N  =  45  the  assigned 
probabilities  were  P5  =  =  P7  -  Pgs  *25,  with  the  remaining  probabilities 

equal  to  zero,  For  these  prior  probabililies  it  can  be  easily  verified  from 
the  payoff  matrix  in  Figure  7  that  the  following  estimated  expected  gains 
result: 

.  858  for  Random  Search 

.  390  for  Single -Factor 

.381  for  Single -Factor  accelerated 

.920  for  RSM -Variation  I 

.919  for  RSM -Variation  1  accelerated 

.880  for  RSM -Variation  II 

.877  for  RSM -Variation  II  accelerated. 

Thus,  the  choice  of  a  search  technique  would  be  RSM -Variation  I,  or  perhaps 
its  accelerated  counterpart. 


*See  Good  (1965)  for  a  summary  of  various  opinions  in  this  controversy. 

2This  is  known  as  a  Bayes  criterion. 

^For  convenience,  the  numbers  of  the  states  of  Nature  and  those  of  the  search 
techniques  will  be  assumed  to  correspond  to  the  order  in  which  they  are 
listed  in  the  payoff  matrices  of  Figures  5-12, 
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D.  DISCUSSION  OF  RESULTS 


The  accumulated  performance  data  provide*  a  guide  for  (election  of  a 
search  technique  by  the  user  of  the  prototype  "OPTIMIZER"  and  for  eventuaL 
incorporation  of  decision  logic  directly  into  the  "OPTIMIZER".  This  guide 
in,  admittedly,  a  rough  one  because  of  the  two  following  reasons: 

(1)  Only  two  values  of  k  were  considered. 

(2)  Prior  probabilities  relating  to  the  simulation 
characteristics  are  required. 

On  this  latter  point,  it  should  be  noted  that  a  good  practice  to  follow  in  assigning 
subjective  probabilities  is  to  estimate  upper  and  lower  values  for  the  probabilities 
If  this  is  done,  these  values  can  be  used  in  conjunction  with  the  appropriate 
payoff  matrix  to  determine  the  sensitivity  of  the  search  technique  selection 
to  the  variation  in  the  probabilities. 

With  regard  to  the  former  point,  the  results  obtained  for  k  3  30  and  k  s  120 
may,  iu  many  Instances,  be  cautiously  interp, dated  and  extrapolated  to  other 
values  of  k  by  comparing  performance  of  the  search  techniques  graphically. 

As  a  simple  illustration,  assume  a  simulation  user  who  has  a  maximum  of  90 
computer  runs  available  is  certain  that  his  simulation,  which  involves  60 
controllable  factors,  can  be  categorized  by  a  unimodal  response  surface 
(i.e.  ,  no  local  maxima),  by  a  starting  point  far  from  the  true  optimum  point, 
and  by  large  random  error.  Furthermore,  assume  that  he  regards  low 
activity  or  high  activity  as  equally  likely.  In  terms  oi  the  prior  probabilities 
assigned  to  the  states  of  Nature,  his  assignment  would  be  p^  =  Pg  =■  ,  5,  with  all 
the  other  p^'s  equal  to  zero.  A  plot  of  gain  for  each  of  the  two  values  of  k  is 
given  in  Figure  13. 

From  this  figure,  it  can  be  seen  that  the  only  two  contending  search  tech¬ 
niques  given  the  assigned  prior  probabilities  are  RSM -Variation  I  and  Random 
Search,^  Also,  the  user  might  reason  that  for  both  k  =  30  and  k  -  120,  the 


*This  suggestion  is  given  by  Good  (1965),  a  firm  believer  in  the  usefulness 
of  subjective  probabilities. 

^The  accelerated  techniques  are  not  plotted  because  they  provide  essentially 
the  same  results  as  their  nonaccelerated  counterparts. 
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best  search  technique  when  N  =  1, 5  k  is  RSM -Variation  I,  and  thus  for  k  =  60 
and  N  *  90  »  1.5  k,  the  best  search  technique  would  also  be  RSM 'Variation  1. 

It  should  be  noted,  however,  that  for  k  equal  to  30,  60,  or  120,  the  corresponding 
fractional  factorial  would  require  only  a  few  runs  more  than  k.  If,  for  instance, 
k  s  64,  the  corresponding  fractional  factorial  would  require  twice  this  number  of 
runs.  This  difficulty  can  be  bypassed,  at  least  for  k  <  1.00,.  by  including  the 
designs  given  by  Plackett  and  Burman  (1946)  in  the  "OPTIMIZER".  These  designs, 
like  the  fractional  factorials,  Involve  only  the  values  ±1  in  terms  of  the  coded 
factors.  Unlike  the  fractional  factorials,  the  Plackett -Burman  designs  never 
require  more  than  k  +  4  computer  runs.  Although  these  designs1  are  not,  in 
general,  straightforward  to  construct,  the  relatively  good  performance  of 
RSM-Variation  I  implies  that  they  should  definitely  be  incorporated  into  a  final 
production  version  of  the  "OPTIMIZER". 

It  can  be  seen  from  the  payoff  matrices  in  Figures  5  through  12  that  the 
performance  of  both  Single -Factor  techniques  is  quite  close  to  being  dominated 
by  the  performance  of  other  techniques.  Thus,  for  all  practical  purposes, 
these  two  techniques  may  be  eliminated  from  consideration.  Each  of  the 
remaining  techniques,  however,  has  the  potential  of  providing  the  maximum 
gain,  depending  on  the  assignment  of  prior  probabilities.  Thus,  these  techniques 
should  remain  as  options  within  "OPTIMIZER".  It  should  be  noted,  however, 
the  gains  obtained  for  Random  Search  in  the  empirical  study  are  likely  to  be 
larger  than  in  practical  situations  because  of  the  preferential  definition  of  the 
overall  search  region  discussed  in  Section  III.C.l. 

As  an  example,  if  each  coordinate  of  the  optimum  point  were  in  the  interval 

(-.75,  .75),  the  overall  search  region  would  be  given  by  -!  <  x,  <  1  for  1=1, 

■  k  *  “ 

....  k.  Thus,  the  volume  enclosed  by  this  region  would  be  2  .  If  the  overall 

region  were  increased  so  that  it  were  given  by  -1 .  1  <  x,  <  1.1  for  1  =  1,  ....  k, 

k  m  i  m 

the  volume  enclosed  would  be  (2,2)  ,  The  difference  in  the  two  volumes  is 
extremely  large  for  the  values  of  k  considered  in  the  study.  For  instance, 
if  k  =  120,  2k  4  1 . 33  x  10^  while  (2, 2)^  *  1. 23  x  lO^1 .  As  a  result  of  Increasing 
the  interval  on  each  x.  by  10%  the  corresponding  volume  of  the  overall  search 


JIn  actuality,  the  fractional  factorial  used  for  the  case  k  =  30  is  one  of  the 
family  of  Plackett -Burman  designs. 


region  would  Increase  by  approximately  10,000,0007a  Thus,  aa  the  uncertainty 
in  the  interval  aaaociated  with  each  factor  increaaes  slightly',  the- cor ree ponding 
overall  search  region  Increases  phenomenally,  causing  a  decrease  in  the  efficiency 
of  Random  Search,  It  is  suggested,  therefore,  that  the  overall  search  region  be 
carefully  defined  before  applying  Random  Search,  In  any  event,  it  can  be  seen 
from  the  data  that  Random  Search  should  not  necessarily  be  the  search  technique 
selected,  t  ven  for  a  large  number  of  factors.  Thus,  the  current  study  has 

i  ‘ 

provided  <  seclusions  different  from  those  of  Brooks  (1959)  and  McArthur  (1961), 

As  far  as  the  accelerated  techniques  are  concerned,  they  do  not  appear 
to  provide  extremely  different  results  than  their  nonaccelerated  counterparts, 

In  fact,  paired  -observation  t -tests  on  the  differences  in  the  gain*  provided  by 
each  pair  of  accelerated  and  nonaccelerated  techniques  indicated  no  significant 
differences  in  performance.1  However,  from  a  decision  -theoretic  rather  than 
a  hypothesis  testing  viewpoint,  it  makes  sense  to  consider  both  type  of  techniques, 
since  neither  dominates  the  other. 


hindsight  analysis  indicates  that  while  acceleration  sometimes  resulted  in 
reaching  a  better  response  In  a  fewer  number  of  computer  runs,  there  were 
also  many  Instances  in  which  acceleration  overshot,  and  wasted  computer  runs. 


IV.  APPLICATION  OF  THE  PROTOTYPE  "OPTIMIZER11 


The  potential  interface  with  "OPTIMIZER"  should,  if  possible,  be  considered 
during  the  development  of  a  simulation,  ■  E"ery  cfx or t  should  be  made  to  minivan 
the  execution  time  of  each  simulation  -U.rvVton,  by  eliminating  unessential 
intermediate  data  analysis  and  summand),  for  example.  The  Input  and  output 
operations  should  be  isolated,  leaving  cnly  a  complete  portion  of  the  model,  or 
a  "kernel.  "  Making  the  kernel  modular  is  highly  desirable,  expecially  if  overlay 
is  necessary  because  of  the  siae  of  the  simulation. 

Ideally,  then,  there  would  be  three  sections  to  every  "OPTIMIZER"  -  simu¬ 
lation  interface.  Section  I  would  contain  the  simulation  input  statements,  logic 
checking  routines,  file  and  table  creations,  a  a  well  as  the  "OPTIMIZER"  input 
statements.  Operations  in  this  section  would  be  independent  of  operations  in 
any  other  sections,  although  data  would  be  exchanged  with  the  other  sections. 
Section  II  would  contain  the  kernel  and  the  "OPTIMIZER"  search  techniques. 

(In  order  to  decrease  core  requirements,  a  "MINI-OPTIMIZER",  i.  e.  ,  one 
selected  search  technique,  could  be  substituted.)  Section  III  would  contain  the 
file  update,  report  generator  portions,  "OPTIMIZER"  summary  output,  etc.  , 
and  would  only  be  executed  after  Section  II  is  completed. 

Military  simulations  generally  require  large  core  storage  requirements 
and  execution  time,  and  are  usually  Input/Output  bound.  It  is  not  uncommon 
for  one  run  to  require  30  minutes  to  totally  execute  while  20  kernel  runs  can 
be  executed  in  the  same  amount  of  time.  Overlay  structure  is  also  quite  common, 
giving  added  impetus  to  the  three  section  structure  as  shown  in  Figure  14. 

However,  many  candidate  simulations  for  optimisation  are  not  under  develop¬ 
ment  now,  but  already  exist.  Hence,  the  interfacing  may  not  be  quite  as  simple, 
although  the  "MINI-OPTIMIZER "/kernel  sectioning  concept  may  still  be  a  good 
one.  An  alternative,  of  course,  is  to  relegate  "OPTIMIZER"  to  a  subroutine 
role,  which  would  require  only  minor  modification  of  "OPTIMIZER.  " 

In  order  to  gain  more  insight  into  the  "OPTIMIZER "-simulation  interface, 
as  well  as  to  examine  the  effectiveness  of  the  prototype  "OPTIMIZER",  it 
seemed  desirable  to  apply  the  "OPTIMIZER"  in  a  realistic  simulation  situation. 
With  the  cooperation  of  the  U.S.  Army  Strategy  and  Tactics  Analysis  Croup 
(STAG),  the  "OPTIMIZER"  was  applied  to  a  full-scale  simulation.  This  test 
case  application  is  described  in  the  following  section. 
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A.  A  TEST  CASE:  FORECAST  II 

FORECAST  II  is  a  on« -sided  expected  value  model  which  attempts  to 
tactically  allocate  finite  resources  against  an  enemy  force.  These  resources 
are  airborne  delivery  and  weapon  systems  combinations  which  are  offensive, 
offensive-defensive  or  defensive  in  nature.  The  enemy  force  is  represented 
by  a  strike  list.  Given  a  strike  list,  the  model  attempts  to  achieve  a  specified 
assurance  of  damage  while  expending  a  minimum  number  of  offensive  sorties, 
thus  allowing  more  targets  to  be  engaged.  The  descriptors  within  the  model 
are  specifically  related  to  the  military  capabilities  of  the  delivery  and  weapon 
systems.  The  input  variables  which  may  be  considered  controllable  by  the 
military  dec  ision  makers  are  those  describing  the  distribution  of  the  delivery 
and  weapon  systems  of  the  friendly  force  and  those  representing  the  friendly 
force's  strategy  in  the  form  of  an  ordered  strike  list. 

The  prototype  "OPTIMIZER"  wao  employed  to  study  the  distribution  of  the 
resources  for  a  fixed  strike  list.  Thus,  the  results  of  the  study  were  unique 
to  the  strike  list  in  effect.  The  figure  of  merit,  selected  to  conform  to  the 
model  action,  was  chosen  as  "the  number  of  targets  attacked  per  sortie  dis¬ 
patched.  "  The  factors  selected  for  optimisation  were  those  parameters 
describing  the  distribution  of  aircraft  to  zones.  Five  aircraft  types  were 
simulated,  of  which  four  types  were  permitted  in  three  base  zones,  while  the 
other  was  allowed  in  only  two  sones.  The  resulting  14  controllable  factors  were 
subject  to  constraints  imposed  by  a  constant  total  inventory  for  each  type  of 
aircraft. 
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1.  Interfacing 


FORECAST  II  is  a  computer  simulation,  written  in  FORTRAN,  which 
fits  the  general  description  previously  attributed  to  military  models;  i.  e. ,  it 
requires  large  amounts  of  core  storage  and  generates  large  amounts  of 
ancillary  information.  Preliminary  discussion  with  the  analyst  who  developed 
FORECAST  II  (a  distinct  advantage  when  attempting  to  interface  "OPTIMIZER" 
with  an  existing  simulation)  led  to  the  conclusion  that  a  kernel  model  Coupled 
with  a  "MINI- OPTIMIZER"  would  provide  the  best  mode  of  interfacing.  It  was 
jointly  decided  that; 

(a)  Unessential  operations  such  as  collateral  damage  assessment, 
normal  output  operations,  etc. ,  should  be  deleted  in  the  kernel  to 
reduce  machine  time  during  execution. 

(b)  The  link  structure  should  be  modified  to  make  additional  core 
storage  available  and  the  "OPTIMIZER"  should  be  modified  to 
delete  unapplicable  options  and  reduce  core  storage  requirements. 

(c)  A  fully  edited  binary  tape  containing  the  strike  list  should  be 
created  in  place  of  a  BCD  tape  in  order  to  decrease  processing 
time. 

These  modifications  and  several  other  refinements  were  made  to  the 
"OPTIMIZER"  and  kernel  in  order  to  bypass  resetting  inventories,  to  calculate 
the  figure  of  merit  directly,  and  to  speed  processing.  The  resulting  computer 
programs,  which  provided  the  required  interface,  were  successfully  executed 
and  used  throughout  the  study. 

2.  Two  Problems 

The  controllable  factors  (the  resource  distribution  parameters)  violated 
the  basic  assumptions  on  which  the  prototype  "OPTIMIZER"  was  constructed. 
These  assumptions  are  that  the  controllable  factors  are  continuous  and  un¬ 
constrained.  Unfortunately,  the  resource  distribution  parameters  were  non¬ 
negative  integer  values  which  had  to  total  to  a  set  integer  value.  It  was  felt  that 
the  violation  of  the  assumption  of  continuity  could  be  ignored  with  little  impact 
on  the  "OPTIMIZER"  results  since  the  magnitude  of  the  total  inventory  was 
sufficiently  large  to  make  transition  between  integers  small. 


However*  the  prototype  "OPTIMIZER"  ia  not  capable  of  handling 
conatrainta,  while  the  controllable  factora  required  two  typea: 

(a)  >  0 

(b)  I  Xj  8  C 

Hence,  it  waa  decided  to  develop  a  reparameterization  in  an  attempt  to  overcome 
the  constraint  difficulty.  The  apecific  reparameterization  selected  ia  illuetrated 
in  the  following  paragraph  for  the  case  where  four  factora  are  involved. 

Conaider  four  continuoua  factors  --  X^,  X2>  Xj.  X4  --  subject  to  the 
above  constraints.  It  should  be  noted  that  if  the  values  of  three  of  these  factora 
are  known,  then  the  fourth  la  automatically  determined  by  conatraint  (b).  Now 
conaider  the  following  re  parameterization; 

Wj  =  In  (Xj/X4) 

W2  =  In  (X2/X4) 

W3  *  In  (X3/X4) 

The  reparameterized  factors  W^,  W^,  and  W3  are  continuous  and  unconstrained. 
The  inverse  of  this  reparameter ization,  together  with  constraint  (b),  results  in: 


X 

X 

X 
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1 
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C  -  (Xj  +  x2  +  x3) 


) 

) 


It  is  thus  possible  to  encode  the  constrained  X  factors,  optimize  in  the  W  factor 
space,  and  decode  the  "optimum"  point  to  obtain  the  values  of  (Xj,  X2>  X3,  X4). 

This  type  of  reparameterization  reduced  the  set  of  14  original  factors 
to  a  aet  of  only  9  new  factora.  Although  dealing  with  the  factors  arising  from 
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the  transcendental  reparameterisation  posed  no  problems  to  the  "OPTIMIZER", 
the  simulation  user  experienced  some  difficulty  In  the  interpretation  of  the 
optimisation  of  the  transformed  factors.  Specifically,  ha  used  a  few  runs  to 
apply  the  Single -Tac  *  search  technique  in  order  to  check  whether  the  kernel 
model,  the  "MIMI-G  7IMIZER",  and  the  interfacing  were  working  properly,  and 
whether  the  constr  .data  were  being  observed.  The  difficulty  arose  because  the 
change  of  a  single  W  factor  causes  simultaneous  changes  in  the  X factors. 
Because  a  simulation  user  would,  in  general,  feel  more  comfortable  with 
optimisation  in  terms  of  factors  which  are  directly  related  to  quantities  in  the 
real  world,  reparameterisation  must  be  regarded  as  a  stopgap  measure,  at  best. 

B.  DISCUSSION  OF  RESULTS 

After  the  interfacing  was  checked  and  the  reparameterisation  programmed 
into  the  "MINI-OPTIMIZER "/kernel,  the  analyst  provided  a  starting  point  which 
he  regarded  as  a  "pretty  good"  allocation.  After  the  step  sises  and  the  overall 
search  region  were  selected,  four  searches  were  made.  These  searches  were 
based  on  Random  Search,  Single -Factor,  RSM-Variatlon  I,  and  RSM-Variation 
II,  respectively.  Each  search  involved  k«=9  transformed  factors  and  was  per¬ 
mitted  a  maximum  of  N-20  computer  runs. 

1.  Performance  of  the  Search  Techniques 

The  specified  starting  point  provided  an  observed  response  of  .  397. 
Because  the  simulation  is  based  on  an  expected  value  model,  this  observed 
response  is  also  the  true  response  at  that  point.  Thus,  in  terms  of  the  measure 
used  in  the  empirical  studies,  the  relative  gain  provided  by  a  search  technique 
was  given  by: 


[("Optimum"  response  found  t>y  technique)  -  .397  / 

(True  optimum  response)  -  .  397  J 

Because  the  true  optimum  is  unknown,  the  best  that  could  be  done  was  to 
determine  a  lower  bound  for  relative  gain.  This  lower  bound  was  calculated 
using  information  from  the  analyst  who  verified  on  theoretical  grounds  that  the 
figure  of  merit  obtained  in  the  simulation  would  never  be  greater  than  .  500. 
This  value  was  used  as  the  true  optimum  response  in  the  definition  of  relative 
gain,  yielding  the  lower  bound. 
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The  results  of  the  application  of  the  four  search  techniques  to 
FORECAST  II  are  given  in  Figure  15.  As  can  be  noted,  RSM-Variation  I 
provided  the  best  results,  a  relative  gain  of  greater  than  38%  in  just  20  computer 
runs.  Somewhat  surprising  was  the  performance  of  the  Single -Factor  approach 
and  RSM-Variation  II.  On  the  basis  of  results  in  the  empirical  studies,  it  might 
have  been  anticipated  that  Single-Factor  would  not  have  performed  as  well  as  it 
did.  Likewise,  it  might  have  been  anticipated  that  the  performance  of  RSM- 
Variation  II  would  have  been  better.  It  must  be  borne  in  mind,  however,  that 
the  results  of  the  empirical  studies  provided  the  value  of  estimated  expected 
gain  based  on  a  number  of  iterations  of  the  same  search  technique.  In  the 
FORECAST  II  application,  the  results  are  based  on  only  a  single  iteration  of 
each  search  technique. 

% 

As  a  whole,  the  application  of  the  prototype  "OPTIMIZER"  to  FORECAST 
II  proved  a  success.  The  relative  ease  of  interfacing  and  using  the  "OPTIMIZER", 
combined  with  the  increased  figure  of  merit  found  by  the  "OPTIMIZER",  shows 
the  potential  value  of  such  an  externally-controlled  optimization  computer 
program.  However,  it  became  apparent  during  the  "OPTIMIZER"  application 
to  FORECAST  II  that  two  modifications  to  "OPTIMIZER"  were  extremely 
desirable. 

2.  Two  Indicated  Modifications  to  "OPTIMIZER" 

Experience  with  application  of  the  prototype  "OPTIMIZER"  to  FORECAST 
II  revealed  that  a  modification  to  permit  constrained  optimization  problems  and 
one  to  provide  a  restart  capability  would  provide  a  more  powerful  "OPTIMIZER.  " 

To  deal  with  constraints  using  the  prototype  "OPTIMIZER",  it  may  be 
possible  to  change  the  problem  into  an  unconstrained  form  either  by  means  of 
an  appropriate  reparameterization  on  the  controllable  factors  or  by  modifying 
the  simulation  to  provide  a  "bad"  figure  of  merit  value  for  any  combination  of 
controllable  factors  which  violates  the  constraints.  *  However,  neither  of  these 
procedures  guarantee  a  solution  to  the  constraint  problem  and,  even  if  successful, 
are  at  best  stopgap  measures  which  levy  additional  requirements  on  the  user  of 


1 


Both  of  these  stratagems  appeared  to  be  reasonably  acceptable  in  the  application 
of  the  "OPTIMIZER"  to  FORECAST  II. 
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FI 6.  15  fUESSLTS  IF  imiCATIM  IF  PHTITTPE  “•PTIMZEr 


the  "OPTIMIZER,  "  Thus,  "OPTIMIZER"  should  be  modified  to  allow  a  reason¬ 
able  number  of  constraints  to  be  handled  without  any  intervention  of  the  user, 
except  for  apecification  of  the  constraints  as  data  input. 

The  "OPTIMIZER"  logic  was  designed  to  make  the  most  efficient  use  of 
the  available  number  of  computer  runs.  This  logic  was  based  on  the  hypothesis 
that  the  user  would  assign  to  the  "OPTIMIZER"  the  maximum  number  of  com¬ 
puter  runs  that  he  could  justify.  For  example,  assume  that  one  of  the  RSM 
techniques  is  used  and  a  point  is  reached  where  a  new  experimental  design  is 
required  to  determine  a  new  path  of  steepest  ascent.  If  not  enough  computer 
runs  remain  to  permit  a  full  design  involving  the  original  k  controllable  factors, 
the  "OPTIMIZER"  uses  a  full  design  with  only  a  selected  subset  of  the  k  factors, 
instead  of  beginning  a  portion  of  the  design  involving  all  of  the  controllable 
factors.  This  procedure  permits  a  path  of  steepest  ascent  (in  a  space  of  reduced 
dimensionality)  to  be  calculated  and  followed. 

Based  on  the  "OPTIMIZER"  application  to  FORECAST  II,  it  is  apparent 
that  in  many  situations  the  user  may  not  be  able  to  (or  may  not  wish  to)  use  the 
total  number  of  available  runs  in  one  "OPTIMIZER"  run.  Thus,  the  user  might 
desire  to  restart  the  "OPTIMIZER"  from  the  point  which  its  original  application 
found  to  be  "optimum"  within  the  number  of  computer  runs  originally  specified. 
The  current  version  of  the  "OPTIMIZER"  does  not  permit  such  a  continuation, 

In  order  to  provide  efficient  restart  capabilities,  the  "OPTIMIZER" 
should  be  modified  so  that  it  "remembers"  the  phase  of  the  "optimisation" 
process  in  progress  when  the  specified  number  of  computer  runs  was  exhausted. 
This  modification  would  assure  that  all  pertinent  data  (such  as  maximum  response 
observed,  location  of  that  response,  parameters  determining  the  path  of  steepest 
ascent)  are  available  should  the  user  desire  to  continue  the  "optimisation" 
process  from  where  it  had  originally  terminated. 
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V.  GONC  LUSIONS 


In  summary,  the  results  of  the  HRB-Singer  study  to  date  have  been  three¬ 
fold: 

(1)  A  prototype  "OPTIMIZER"  exists. 

(2)  Information  to  aid  in  thb  selection  of  a  search  technique  is  available. 

(3)  Preliminary  application  of  the  prototype  "OPTIMIZER"  has  given 
positive  results. 

At  this  stage  of  the  study,  It  seems  plausible  that  the  "OPTIMIZER"  may  prove 
of  great  value  to  the  simulation  user  and,  hence,  its  development  should  be 
continued. 

Despite  agrowing  literature*  which  touches  on  the  topic  of  optimization  in 
the  simulation  framework,  most  practical  applications  tend  to  rely  on  trial- 
and-error  procedures  or  on  the  relatively  inefficient  Single-Factor  approach. 
Thus,  the  primary  utility  of  the  "OPTIMIZER"  will  be  that  it  provides  the  simu¬ 
lation  user  with  a  means  of  managing  an  efficient  automatic  search  for  an 
optimum. 

In  the  rare  case  where  familiar  processes  are  being  simulated,  where  a 
well-understood  figure  of  merit  is  to  be  optimized,  and  where  only  a  few 
factors  affect  this  figure  of  merit,  the  simulation  user  may  be  able  to  do  a 
fairly  good  job  of  optimizing  by  heuristic  means.  In  essence,  he  would  select 
initial  values  for  the  controllable  factors,  run  the  simulation  for  these  values, 
observe  intermediate  output  and  the  figure  of  me’-it  obtained,  use  this  data  to 
form  tentative  hypotheses  about  how  the  values  of  the  controllable  factors  should 
be  changed,  change  these  values,  and  keep  repeating  this  procedure.  Needless 
to  *  *y,  much  effort  would  be  required  of  the  user,  and  the  whole  process  could 
easily  extend  over  weeM  because  of  the  time  required  for  analysis  and  for 
computer  turnaround  betvyeen  the  individual  runs. 


*  For  example,  Ackoff  ( 1962),  Bonini  (1963),  Burdick  and  Naylor  ( 1966),  Conway, 
Johnson,  and  Maxwell  (1959),  Emshoff  and  Sisson  (1970),  Hunter  and  Naylor  (1970) 
Jacoby  and  Harrison  ( 1962),  Mihram  (1970),  Naylor  (1969),  Naylor,  Balintfy, 
Burdick,  and  Chu  (1966).  Overholt  (1970),  Smith  (1969),  Smith  ( 1970a),  and 
Smith  (1970b). 
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Application  of  the  "OPTIMIZER",  on  the  other  hand,  would  require  little 
effort  on  the  part  of  the  user,  except  for  the  initial  interfacing.  Further,  by 
automating  intermediate  analyse* ,  and  thereby  consolidating  the  many  runs 
otherwise  necessary,  the  "OPTIMIZER"  would  significantly  reduce  the  overall 
time  involved.  Therefore,  even  in  this  situation,  the  "OPTIMIZER"  has 
potential  usefulness. 

As  an  alternative  to  an  "OPTIMIZER"  computer  program,  a  set  of  search 
technique  alogrithms  could  be  provided  to  the  simulation  user.  However,  the 
fairly  tedious  steps  in  some  of  these  algorithms,  coupled  with /the  fact  that 
simulation  input  would  have  to  be  manually  changed  between  each  computer  run, 
might  prove  discouraging  to  the  user,  causing  him  to  abandon  the  algorithms  in 
favor  of  an  easier,  but  less  efficient  approach.  If,  in  fact,  the  user  did  persevere 
with  the  algorithms,  it  is  reasonable  to  assume  that  he  would  write  his  own 
program  to  carry  out  the  search  technique  steps  and  to  permit  automatic  modi¬ 
fication  of  simulation  inputs.  Thus,  supplying  the  simulation  user  wltl^  only  the 

1 

algorithms  appears  highly  inefficient  compared  to  providing  him  with  a\proven 
"OPTIMIZER"  computer  package.  \ 

It  must  be  noted  that  further  work  remains  to  be  done  before  the  "OPTIMIZER" 
can  be  used  as  an  effective  tool  in  practical  applications.  For  example,  the 
prototype  should  be  modified  to  provide  the  restart  and  constrained  optimisation 
capabilities  discussed  in  the  previous  chapter.  In  addition,  further  investigative 
applications  of  the  "OPTIMIZER"  to  full-scale  computer  simulations  should  be 
made  in  order  to  provide  information  for  a  detailed  critique  of  Us  overall  perfor¬ 
mance. 

Nonetheless,  there  are  strong  indications  at  the  present  time  that  the 
"OPTIMIZER"  should  prove  valuable  to  the  simulation  user. 
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ABSTRACT 

Exparlaental  design  technique*  are  used  throughout  the 
eeleneea  In  the  analyele  of  etatieticel  processes.  Thee*  techniques 
can  also  be  used  to  analyse  large ,  essentially  deterministic,  soaplex 
systems.  In  this  non-seatlstioal  case,  they  are  not  neneeearlly 
the  noet  efficient  tools. 

In  the  present  paper  we  describe  a  method  which  was  developed 
to  aid  in  the  investigation  of  a  large  deterministic  computer  simula¬ 
tion.  This  Method  would  sees  to  he  applicable  to  the  study  of  many 
complex  non-stetlstlcal  eye tees.  Me  include  possible  extensions  of  the 
method  as  well  as  several  alternate  approaches  to  the  general  problem, 
further  work  to  extend  this  technique  to  certain  statistical  ays tens  is 
planned. 


INTRODUCTION 

Experimental  design  Involves  the  analysis  of  a  complex  system 
by  choosing  e  ssrlss  of  sxperimonts  to  perform  on  the  system.  Techniques 
for  anslysls  of  stetlsticel  processae  have  been  developed  Including 
factorial,  fractional  factorial,  end  other  such  designs.  (Rsferencs 
[1]  by  Huntsr  end  Naylor  gives  e  good  summery  of  these.)  For  systems 
with  more  then  a  few  Inputs  or  for  which  experiments  tend  to  be  expansive 
a  random  search  often  gives  the  most  efficient  design,  and  Its  efficiency 
can  bs  quite  low,  (See  McArthur  [3].) 

In  thla  paper  we  describe  e  technique  (called  the  STAG  Monotone 
Experimental  Design  Algorithm  (SMEDAL))  for  analysing  deterministic, 
rather  then  stetlsticel,  systems  which  exhibit  certain  properties  (described 
In  section  I).  SMEDAL  was  originally  developed  to  eld  In  the  analysis 
of  computer  simulations  end  seems  to  be  applicable  to  many  systema  in  which 
statistical  variation  plays  s  minor,  or  non-txlstsnf.  rola.  Extension  of 
this  tachnlqus  to  statistics!  systems  Is  plennsd. 


The  remainder  of  this  paper  has  been  photographically  reproduced  from  the 
manuscript  submitted  by  the  author. 
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SECT1  OH  1 


smmisssm 

We  heve  a  complex  system  with  discrete  Inputs  and  outputs. 

Wo  vlah  to  analyse  the  relations  between  inputs  and  outputs  with 
the  general  purpose  being  to  determine  the  response  of  the  system 
using  the  smallest  number  of  actual  experiments  possible.  Specific 
relations  between  inputs  are  considered  and  their  usefulness  to 
this  program  evaluated. 

SPECIFIC  PROBLEM 

The  ayatom  take*  n  inputs  where  input  1  has  levels.  Its 
output  is  v  where  v  »  0,1,2,  .  .  .  k.  figure  1.) 


Figure  1 


input  1 
input  2 

« 

input  n 


1  <_  input  i  <.  mjt 


0  <  v  *;  k 


A  aet  of  inputs  for  one  experiment  will  be  considered  to  be 
an  n-tuplc  (x^,  .  .  .  ,xn)  with  1  <  xA  <_  for  all  i.  We  will 
say  that  N(xj,  .  .  .  ,xn)  «  v  if  experiment  (x^,  .  .  .  ,xn>  yields 
output  v.  Therefore  N  is  a  function  from  the 
n-tuples  to  the  net  {0,1,2,  .  .  .  ,k}  . 

The  first  relation  between  inputs  is  the  monotone  relation. 
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This  relation  states  that  1£  all  inputs  but  ons  ars  held  tlxed, 
then  an  increase  in  the  unfixed  input  cannot  cause  a  decrease  in 
the  output.  Xn  symbols i 

It  N(xi,  .  .  .  ,xn)  -  v,  then  H<Xj,  .  .  .  ,xt+i,  .  .  .  xn) 

to v  all  i  and  N(x^i  ...  ,x^*l,  ...  ^ v  for  ell  1. 

Example;  Let  n  «  8  and  ni  *  3  for  all  i.  Let  k  -  l.  If 

N(2,2,.  .  ,  ,2)  -  1,  then  N(3,2,2,  .  .  .  ,2)  -  1.  In  fact,  If 

(xj . xn)  is  any  combination  of  2's  and  3's,  then 

N(xi . xn)  «  1.  If  N(2,2,  .  .  .  (2)  "  0,  eisdlar  results 

follow  as  above  with  "1"  substituted  for  "3".  Thus  it  follows 
that  N(l,2,2 . 2)  -  0,  etc. 

In  tho  above  example,  if  N(2,2,.  ,,,{)■  0,  thnn  the 
monotone  relation  telle  nothing  about  N(l,3,2,  ...  ,2).  Ono 
input  has  been  raised  and  another  lowered;  monotonicity  does  not 
apply  iu  such  a  case.  Although  much  can  be  done  with  our  problem 
using  only  the  monotones  relation,  it  is  possible  that  there  will 
also  exist  dominance  relations  between  the  positions  of  the  inputs. 
These  dominance  relations  will  occur  when  it  is  uniformly  true 
that  raising  certain  inputs  and  lowering  certain  others  always 
tends  to  have  the  same  affect  on  the  output. 

First,  to  clarify  what  la  meant  by  "position",  consider  the 
8-tuple  (3, 2, 1,1, 1,1, 1,1),  3  is  In  the  first  position,  2  is  in  the 

second  position,  and  l's  are  in  positions  3  through  8. 

Say  that  position  i  dominates  position  J  if  an  increase  of  1 
In  position  1  with  a  decrease  of  1  in  position  j  cannot  cause  a 
decrease  in  tho  output.  In  symbols; 


Position  1  dominates  position  J  nssns  If  H(xj,  .  .  .  .xn)  ■  v, 
thsn  N(xj ,  .  .  ,«  i  x^t-l,  *  i  i  i  xj-1 ,  m  .  •  ,Xn)  >  v  snd 
N(x^t  *.*  *  $  x^pl |  •  •.  •  ,  xj+1 »  ■  •  i  <  v  ■  • 

This  rslatlon  could  occur  bstwosn  any  two  positions.  The 
mors  such  relations  that  exist,  the  more  information  that  can  be 
obtained. 

Example!  If  position  1  dominates  position  2  and  N(l, 3, 3, 2, 2, 1,1,1)  ■  1 
then  N(2, 2, 3 ,2, 2 ,1,1,1)  ■  1  (using  parameters  of  previous  example). 

SECTION  II 


OUTLINE  2E  2115  ALGORITHM 

A  detailed  description  of  SMEDAL,  with  possible  extensions, 
is  given  In  tha  following  sections.  Included  here  is  a  basic 
outline  of  the  algorithm  and  its  use. 

Consider  two  computer  programs  called,  for  the  purposes  of 
this  paper,  P  and  Q. 

1.  P  has  as  inputs:  (1)  the  dominance  relations  for  the 
specific  system  under  study  (There  possibly  are  none.),  (2)  a 
set  SjL  of  n-tuplas  (representing  those  tuples  c  for  which  N(c) 

1b  not  known),  and  (3)  tuple  with  its  outcome,  N(a^+j)  . 

(The  tuple  represents  the  last  experiment  run  on  the  system; 

N(ai+l)  has  Just  been  determined.)  P  has  for  output  the  set 
®i+l  "  ®i  "  ^i+1  where  consists  of  all  n-tuples  bj_,  .  .  .  ,br 

in  Sj.  for  which  N(n^.^)  implies  N(b}),  .  .  .  ,N(br) ,  using 


-334' 


Ch«  mono ton*  and  dooinanca  rolationa.  Thua  contain* 

everything  in  8^  that  la  not  in  fy+j  .  (Saa  figure  2.) 


dooinanca  relations 


sat  Sf  of  tuples 
•till  undatanalnad 


tuple  a1+l  with  H(a1+1) 


8i+1  ,  tha  new 
■•to!  tuples 
•till  undetermined 


Figure  2 

2.  Q  has  as  Inputs  tha  dooinanca  relations  (if  any)  and  the 
•at  Sj  of  tuples  still  undetermined .  Q'a  output  is  one  tuple, 
*J+1  *ron  SJ  *  whare  •j+i  i«  chosan  so  that  H(aj+1) ,  whether 
*  0  or  1  or  .  ,  ,  or  k,  will  determine  the  maximum  number 
of  tuples  x  in  Sj  .  (See  figure  3.) 


dominance  relations 


set  Sj  of  tuples 
atlll  undetermined 


tuple  aJ+1  , 
the  "best"  tuple 
to  run  next 


Figure  3 


Using  P  and  Q  ,  tha  algorithm  works  as  follows:  (See 
figure  4,  next  page.) 

(1)  With  S  *  all  tuples,  run  Q  to  get  a^. 

(2)  Run  S}  .  Find  N(a^)  ,  (first  oxpariment) 
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(3)  With  S  ■  all  tuple*  and  N(aj)  ,  tun  P  to  gat 

Sj  *  S  -  Kj  ,  where  -  set  of  tuplaa  b^,  .  .  .  ,bf  for  which 
M(a^)  implies  N(b^) . N(br). 

(4)  With  S}  ,  run  Q  to  gat  a2. 

(5)  Run  *2  *  Find  N(a2)  .  (aacond  experiment) 

(6)  With  and  M(a2>  ,  run  P  to  got  S2  ■  Sj  «  Kj, 
where  K2  is  as  above. 

(7)  With  S2  ,  run  Q  to  get  a^  . 


(8)  Continue  until  all  tuples  are  determined  or  further 
experiments  aro  uneconomical. 


-[Q) - *-a  SYSTEM) 

N(ai) 

s— HPJ 


l1-_»[Q)--^a2—[  SYSTEM) 

N(a,) 

\ 

— — [PJ 

I 


S2—  [Q 


Figure  4 

SMEDM,  LOGIC 
FLOW  DIAGRAM 


‘3 


[SYSTEM] 


\ 
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A  number  of  special  cates  of  SMEDAL  have  been  run  to  help 
estimate  tie  efficiency  of  the  algorithm.  In  general,  the  number 
of  system  experiments  vas  reduced  to  lesa  than  20%  of  the  total 
number  to  get  all  outcomes,  and  to  a  much  smaller  porcent  to  get 
lets  than  all  outcomes. 

The  following  table  summarises  some  of  these  runs.  The  case 
of  6  inputs  at  3  levels  each,  with  two  outputs  (  0  and  1  ),  was 
chosen  as  representative,  although  other  cases  have  been  run  with 
comparable  results. 

TABLE 

6  inputs  at  3  levels  each,  2  outputs 

729  outcomes  to  determine  (total) 

For  «Mch  case,  the  values  N(a^)  for  i  ■  1,2,  .  .  .  were 
chosen  at  random.  Q(w,ie)  with  w  ■  2  and  s  -  .3  was  used.  (See 
Section  III,  part  D.) 


Case  Number 


Number  of  System 
Experiments  made 


12 

19 

22 

33 


Percent  of  Number  of 
System  Experiments  made 
Over  total  number 


1.6% 

2.6% 

3.0% 

4.5% 

8.9% 

19.2% 


Percent  of  all 

Outcomes 

Determined 


50% 

60% 

70% 

80% 

90% 

100% 


65 


SECTION  III 


F  am  o  FOR  k  ■  i 

This  section  contains  the  details  of  programs  P  and  Q, 
whose  use  is  described  In  section  II , for the  case  k  "l.  In 
this  case,  the  output  of  the  system  is  0  or  1  , 

1AB3LA 

First  it  will  be  helpful  to  make  some  definitions  and 
establish  some  propositions. 

1.  Definition.  Let  a  be  an  n-tupla.  Then  there  are  collections 
of  n-tup les  { bj,  .  .  .  ,br  }  and  (c^,  .  .  .  ,c#  )  such  that 

if  N(a)  ■  1,  then  hy  the  monotone  relation  N(b^)  ■  1  for 
i  «>  1,2,  ,  .  ,  ,r  ,  and  if  N(a)  =  0,  then  by  the  monotone  relation 
N(cj)  ■  0  for  J  »  1,2,  ,  .  . ,s.  Call  r+1  "the  number  of  outcomes 
determined  if  a  wins"  and  denote  it  by  #(a).  Call  s+1  "the 
number  of  outcomes  determined  if  a  loses"  and  denote  It  by 

Example:  If  the  system  has  8  inputs  at  three  levels  each, 

then  #(22222222)  -  25C  and  #'(12222222)  -  128.  (Note:  tuples 
(2, 2, 2,2, 2,2,2, 2)  will  be  written  (22222222)  from  now  on.) 

2.  Definition.  I,ct  x  -  (xi,  .  .  .  ,xn)  and  y  -  (y^,  .  .  .  ,yn) 
be  tuples.  Then  lub(x,y)  ■  z  where  z  -  (z^,  .  .  .  ,*„)  with 

z^  •>  max(x£  ,yj )  ,  and  glb(x,y)  ®  w  where  w  *■  (wi . wn) 

with  w^  »  min(x^  ,y^)  . 
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Example:  Let  x  «=  (31222222)  and  y  -  (22212222) .  Then 
lub(x,y)  “  (32222222)  and  glb(x,y)  ■  (21212222). 

3.  groPPflUftPn.  #<*  «nd  y)  -  #(x)  +  #(y)  .  #(lub(x,y))  and 
#'(x  ind  y)  -  #'(x)  ♦  #'(>)  -  #'(Blb(x,y))  where  #(x  and  y) 

la  the  tuunbjr  of  outcomea  determined  if  H(x)  -  1  and  N(y)  ■  1. 
Proof:  Immediate  from  daflnitlone  and  monotone  relation. 

Example:  Suppoae  N<22222222)  «  1  and  N(31222222)  -  1. 

Since  #(22222222)  -  256  and  #(31222222)  -  192,  one  might  gueaa 
(or  hope)  that  knowing  N(22222222)  -  1  and  N(31222222)  -  1  would 
give  256  +  192  -  448  outcomes.  However,  this  is  not  true. 

Since  lub((22222222), (31222222))  -  (32222222)  and  #(32222222)  -  128, 
proposition  3  implies  we  have  only  256  +  192  -  128  «  320  outcomes. 

4.  We  will  new  describe  program  I>.  For  simplicity,  assume  (for 
the  moment)  that  there  ore  no  dominance  relations.  To  start, 

P  will  print  table  T0  ,  which  consists  of  rows  of  the  form: 
x  ,  #(x)  ,  #'(x)  for  all  tuples  x.  In  this  case  #(x)  and 
#'(x)  are  easy  to  compute  by  a  simple  application  of  monotonicity. 

Example:  Assume  8  inputs  at  3  levels  each.  Then  #(22222222)  - 

g 

2  “  256  ,  #(32222222)  -  27  -  128  ,  #(12222222)  *=  3-27  =  384 
#'(22222222)  «  28  ~  256  ,  #'(32222222)  -  3‘27  »  384, 

#'(12222222)  -  27  «  128  ,  #01222222)  -  3-26  -  192,  and 
#'(11322222)  -  3* 25  -  96. 

After  program  q  has  chosen  ax  and  N(a1)  has  been  determined, 
P  prints  tabic  Tp  which  consists  of  rows  of  the  form: 
x  ,  #i(x)  ,  #'x(x)  where  #L(x)  and  #' L(x)  are  computed  as  follows: 
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If  N^)  -  0,  then  #j(x)  -  #<x)  and  #'l(x>  «'  #'<x)  -  #' (glb(a1|X)) . 

If  N(aj)  ■  1,  then  tf^(x)  ■  0(x)  -  #(lub(«i',x})  and  #' ^(x)  ■  #'(x). 

In  general,  after  Q  has  chosen  at  and  N(aj)  has  been 
determined,  P  prints  table  Tj  which  consists  of  rows  of  the  form: 

x  ,  #^(x)  ,  0'i<x)  where:  If  N(a^)  ■  0,  then  #^(x)  »  ^i-l(*) 
and  -  #'1.l<x)  -  #'i.1(glb(ai ,x)> .  If  N(a1>  -  1,  then 

#t(x)  “  #j_i(x)  -  ^^^ClubCajjx))  and  0* ^(x)  -  #'i,.i(x). 

5.  It  Is  easy  to  verify  that  the  rows  of  table  give:  .tuple  x, 
the  number  of  outcomes  determined  if  x  wins,  the  number  of  outcomes 
determined  if  x  loses,  given  that  a^,  .  .  ,  ,a^  have  been  choBen 
and  N(ai),  ,  ,  .  ,N(a^)  have  been  already  determined.  (If  N(x) 
is  determined  by  ar,  then  either  (x)  or  #'j(x)  will  be  0  for 
J  >r.  These  tuples  x  will  be  automatically  oxcluded  from  further 

I 

consideration,  if  the  method  for  computing  the  T^  described  above 
is  used.  See  Part  C  concerning  program  Q.  Set  S^,  referred  to 
in  section  II,  consists  of  those  tuples  x  for  which  either  #^(x) 
or  #'j(x)  equals  0.)  Note  that  proposition  3  implies:  if  Nfa^)  »  1 
then  #j(x)  «=■  i?(x  end  a^)  -  #(aj)  *=  #(x)  -  #(lub(a^,x)) ,  and  clearly 
this  formula  iterates  with  and  replacing  #i  and  aj_. 

It  is  possible  to  write  down  a  non-itorative  formula  for 
#^(x)  using  i0  nested  summations  and  the  i^1  generalization  of 
proposition  3  where  i0  is  the  number  of  aj  such  that  #(«j)  «  1  for 
J  <  i,  but  i:hcre.  seems  to  be  no  real  use  lor  this  generalized  formula 
considering  the  sequential  nature  of  Still  DAL. 


PART  B 


Mow  we  will  consider  how  P  is  to  bo  adjusted  to  take  into 
account  dominance  relations.  Pirst,  some  definitions  and  their 
consequences  will  be  stated. 

6..  Definition.  Let  x  and  y  be  tuples.  Me  say  "x  dominates  y" 
if  N(x)  >.  N<y).  In  this  case  we  write  x  y.  (Note:  "»"  is 
a  partial  order  on  the  set  of  tuples.) 

7.  Proposition,  x  »  y  if  and  only  if  N(y)  “  1  implies  N(x)  ■  1 
and  N(x)  ■  0  Implies  N(y)  ■  0. 

8.  Proposition,  x  »  y  if  xj.  >_  y^  for  all  i. 

9.  Proposition.  If  position  i  dominates  position  J  (terminology  from 
section  I),  xr  >_  yr  for  all  r  i*  i,J  ,  and  x^-1  >.yi,  Xyl-1  iyj.then 

x  »  y . 

Proof:  We  ub«  proposition  7.  Suppose  N(y)  ■  1.  Then 
1  ■  N(y]_,  .  .  ,  *yn)  11  (yi »  ...  »xj_-l*  .  »  •  ,xj+l ,  .  •  .  ,yn) 

N(yj^ >  ...  ...  |Xj ....  £  N(xj ,  .  .  .  ....  ,xj  >  •  . 

-  N(x).  Thus  N(x)  ■  l.  Similarly,  H(x)  »  0  implies  N(y)  *  0. 

10.  Proposition.  Suppose  positions  i^,  .  .  .  ,im  dominate 
respectively  .  ,  .  ,Jm,  (Several  ir  and  Jr  may  be  repeated.) 

If  Xir-1  >_yir  and  xj^+li  yjr  f°r  *  "  1»2,  .  .  .  ,m  and  x^^, 
otherwise,  then  x  »y. 

Proof:  Apply  proposition  9  m  times. 

Example:  Suppose  position  1  dominates  position  2.  Then 
(31222222)  »  (22222222)  and  (22222222)  »  (13222222).  However, 
it  con  not  be  concluded  that  (22222222)  »  (23222222)  or 
(21222222)  »  (13222222). 
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Now,  if  there  are  dominance  relations,  P  <pill  work  as  follows: 

a^  has  been  chosen  and  N(e^)  has  been  determined.  is  known. 

P  is  to  compute  T^.  First,  let  D^  be  the  set  of  tuples  formed  as 
follows:  If  r  dominates  sand  N(ai)  ■  1,  form 
(®1*  •  *  •  *®r+^*  •  •  •  iag”l»  •  •  • isn) » 

(ax,  •  .  .  ,ar+2 ,  ,  .  .  ,ag-2,  .  .  . ,an) ,  etc.  If  r  dominates  s 

and  NUp  -  0,  form  (aj ,  .  .  .  ,ar«l,  .  .  .  ,a8+l,  .  .  .  ,an), 

(aj,  ...  |Sjkw2 ,  ...  ,dg>f2 ,  ...  ,an) ,  etc.  Do  this  for  each 
dominance  relation  r  over  s.  (One  at  a  time.)  is  the  set  of 
all  tuples  constructed  this  way,. 

Examples:  4  inputs  at  3  levels.  (1)  a^  -  (2222),  N(a1)  «  l, 
and  1  dominates  3.  D1  -  {(3212)}  .  (2)  -  (2222),  N(aJl)  -  1, 

1  dominates  3,  and  2  dominates  3.  D^  *  {  (3212),  (2312)}  . 

Now  N(ax)  “  N(dj)  for  all  dj  in  Use  P  with  a^  and  N(a^) 
to  get  table  T^g  ■  without  dominance  as  in  Part  A  above.  Use 
P  with  dx  and  N(dx)  to  get  table  *  table  computed  from  T^q  as 
in  Part  A.  Continue  until  finally  T^t  «»  table  computed  from 
^it-1  as  in  Part  A  with  rl £  and  N(dt)  where  dL  is  the  last  element 
of  D^.  Thou  let  ■  T^t* 

PAUTC 

Program  Q  chooses  a^  from  table  as  follows:  (1)  Let 
S  •=  {x  :  min(#i_1(x)  ,#'t.j(x))  min(#1_1(y) '^.jfy))  for  all  y  ) 

(2)  Lot  S»-  {  x  !  x  in  S  and  mox^^.^x)  ,#'t.j(x)) 

mox(#X_l(y) ,#'i.x(y))  for  all  y  in  S  ) 
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(3)  «  on*  of  Che  dements  of  S  (arbitrary  choice) 

Thus  a^  is  chosen  so  that,  whether  it  win*  or  loses  (i.e. 
whether  H(oi)  -  1  or  0) ,  it  is  sure  to  determine  at  least  as  many 
outcomes  as  any  ether  tuple,  and,  of  all  those  tuples  with  this 
property,  no  other  can  determine  more.  . 

PART  P 

With  P  and  Q  as  described  above,  the  graph  of  tuple  x 
vs.  the  number  of  outcomes  determined  by  x  has  the  form  shown  in  figure  3, 

number  of  outcomes 
determined  by  x 

x 

figure  3 

For  investigations  in  which  there  will  be  relatively  few 

actual  experiments  on  the  system  (e.g.  20  out  of  G5G1),  this  may 

be  quite  satisfactory,  since  the  low  part  of  the  curve  will  never 

be  reached.  However,  when  more  experiments  ere  to  be  run  (eay  100 

out  of  6361),  it  will  be  preferable  for  the  curve  to  have  the 
form  shown  In  figure  6. 

number  of  outcomes 
determined  by  x 

x 

figure  6 
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The  program  Q(w,z),  which  is  a  modification  of  Q,  can  be 
uaed  in  thii  aituation.  Tlie  parameters  w  and  «  depend  on  the 
total  number  of  experiments  possible  on  the  system  and  the  number 
of  experiments  which  are  to  be  run. 

Parameter  w  ia  the  number  of  outcomes  that  the  running  of  one 
tuple  x  should  determine,  Good  results  have  been  obtained  with 
w  ■  2  and  w  ■  4.  Parameter  a  la  a  number  greater  than  0  and  less 
than  or  equal  to  1,  When  a  ■  1,  Q(w,i)  ■  Q.  Roughly,  a  indicates 
the  elope  desired  in  figure  2,  with  the  Blopo  nearer  to  zero  as 
z  gets  small.  Good  results  have  been  obtained  with  z  ■  .3  and 
a  •  .5  . 

Q(w,z)  chooses  from  table  as  foYlGRj;  (1)  Using  Q, 

choose  e'i  ns  usual.  (2)  Using  Q  again  and  rejecting  those  tuples  x 
such  that  tnln(#£_1(x),#,i„i(x))  >  z*min(#l.1(a't) * 
choose  a^,  (3)  If  min(#i.^<ai) <  w,  then  redefine 

a^  to  be  a'^,  the  original  choice  of  Q.  If  min(#j_j(aj )  ,#'^^(11^ )) 
la  still  less  than  w,  terminate  SMEUAL. 

PART  E 

A  different  refinement  of  program  Q  ia  possible  via  a 
heuristic  search  algorithm.  This  is  discussed  in  section  V, 
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SECTION  IV 


P  AMD  0  FOR  k  > I 

We  Mill  outline  the  change*  to  bo  made  In  P  and  Q  if  k  ■  2. 

(outputs  0,1,  and  2)  The  generalisation  to  other  caaea  will  be 
analogous. 

PROGRAM  P 

Tha  table  will  consist  of  entries  of  the  form: 
x,  (x)  ,  #2^(x)  where  #Vi<x)  ■  the  number  of 

outcomes  determined  if  N(x)  *  v  for  v  -  0,1,2. 

Given  Ti_1,  at ,  and  N(a^) ,  table  is  computed  as  follows: 

(1)  If  N(a  )  -  0,  then  l?°i(x)  «  #°i-l(x)  -  #°i-l(glb(a1,x>) ,  and 

#Vi(x)  *  ^Vi-xW  for  v  -  1,2.  (2)  If  N^)  -  2,  then 

#vi<x)  ■  for  v  -  0,1,  and  ^(x)  -  #2i.i(x>  -  *2i-l(lub^i ,x)). 

(3)  Note:  In  this  section,  z  »  w  will  mean  zj  >.  wj  for  all  J. 

If  N(a^)  **  1  :  First,  if  x  »  a^,  set  #®j(x)  *  0  and  if  x  <<  ai# 

e% 

set  #  i(x)  ■  0.  Otherwise,  set  (x)  -  #vj_x(x)  for  v  ■  0  or  2. 

Second,  when  the  above  is  done  for  all  x,  #^(x)  ■  the  number  of 

tuples  y  such  that  N(y)  is  still  undetermined  and  either  y  »  x  and 

«  0 
#  t(y)  •  0  or  y  «  x  and  #  ^ (y>  ■  0. 

Using  the  procedure  described  above,  a  tuple  is  determined 

when  two  of  its  table  entries  are  zero.  (For  the  k  ■  1  case  (section  III), 

a  tuple  was  determined  when  one  of  its  entries  was  zero.) 
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Concerning  3),  the  N(a£>  ■  1  case,  ^^(x)  ■  the  number  of 
tuples  atill  undetermined  whose  outcomes  would  be  determined  if 
N(x)  ■  1.  Now  N(x)  “  1  gives  information  only  about  those  tuples 
such  that  y  »  x  or  y  «  xj  thus,  N(x)  ■  1  and  y  »  x  implies 
N(y)  i.  1,  and  N(x)  ■  1  and  y  «x  Implies  N(y)  £  1.  Such  y  are 
determined  only  under  the  circumstances  listed  under  3)  above. 

PROGRAM  Q 

The  generalization  to  the  case  k  ■  2  is  Immediate. 

SECTION  V 

EXTENDING  Q_  USING  HEURISTIC  SEARCH  TECHNIQUES 

For  simplicity  aesume  k  ■  1  throughout.  Program  Q  chooses 
as  the  tuple  that  will  gain  at;  least  as  much  as  any  other  tuple 
and  will  gain  no  less  than  any  tuple  with  this  property.  However, 
it  is  possible  that  Q  will  choose  a^  and  a^+j  such  that  each  is  as 
above,  while  there  exist  two  tuples  a'i  and  «'i+1  such  that  if 
these  were  chosen,  the  total  number  of  outcomes  determined  (over 
the  two  steps  i  and  i+1)  will  be  more  than  that  determined  if  a^, 
and  are  chosen. 

For  example,  if  in  table  T^.j^  we  had  #i-l(«i)  ■  7, 

^ * i_ 1 ^ ai )  “  7«  ^i-l(fl'i)  ■  6,  and  #'^_i(x)  ■  6,  then  Q  would 
choose  a^  over  a'^.  Then  it  could  huppen  that  e^i  would  be  the 
best  choice  in  T^  where  (®i+i)  *  3  and  #*j(a^+^)  -  3,  while 


the  boat  choice,  In  T'j,  the  table  formed  if  a'^  is  choaen, 

would  have  Ij/a'i+j}  -  5  and  ■  5.  Clearly,  a'^  and 

**1+1  wuld  bo  a  better  pair  of  tuplae  to  run  (aa  they  determine 
11  aa  compared  to  10  outcomes),  but  Q  would  not  know  thia  beeauae 
It  looka  only,  ao  to  speak,  one  level  ahead  each  time  It  makes  a 
choice.  To  correct  thia,  Q  could  be  extended  uaing  the  following 
hcurlatic  aearch  algorithm,  which  la  patterned  after  that  deacrlbed 
In  Slagle  and  Lee  [4] . 

Assume  we  have  table  ,  and  we  wish  to  chooae  a^,  the 

"beat"  tuple  to  run  next. 

A,  Flrat,  we  describe  a  single  level  algorithm.  (Program  Q) 
Let  MN(x)  ■  minimum  number  of  outcomes  determined  if  x  is  run  ■ 
min(tfi-i(x)  ,#Vi(x)) ,  and  let  MX(x)  -  maximum  number  of  outcomes 
determined  if  x  is  run  -  mnx(^_j(x)  ,#'t  ^(x)) .  Then  a^  is  choaen 
ao  that  MN(a^)  1b  maximal  over  all  x  and  MX(a^)  la  maximal  over  all 
X  auch  that  MN(x)  -  MN(a^) . 

B.  Now  we  decribe  how  to  extend  the  algorithm  to  aeveral 
levels.  Refer  to  figure  7  on  page  20. 

1.  Determine  parameters  N  and  M,  defined  below.  The  values 
for  N  and  M  will  depend  on  such  factors  as  the . resouraes  available 
and  the  purposes  of  the  research. 

2.  Level  1:  Find  the  best  N  tuples  a^,  ,  .  .  ,  a^jj  by 
the  single  level  method. 

3.  Level  2:  For  sll  j  there  are  two  easeai  a^j  wins  and 
ajj  loses.  For  each  of  these  cases,  choose  best  tuple  bjjjj  by  the 


single  leyol  method  where  J  •  1,2,  .  .  .  ,N  end  k  ■  0  correspond* 
to  *£j  wine,  k  -  1  corresponds  to  e^j  loses.  Determine  minimum 
number  of  outcomes  n^ji  *  *i-l(®ip  +  MNCb^j j_>  end 
«2jo  “  ^ 1  i-l(*lj)  +  MN(bijo)«  Let  MN2j  ■  min(n2j i »n2jo) 

MXjj  ■  max(m2ji,m2jo)  •  Now  choose  tuple  aij0  where  MN2j0  ie 
mexlutum  for  ell  j  end  MX2j0  *8  max^nun  f°r  ell  J  such  thet 

mn2J  -  mn2Jo. 

Compere  MN2j0  with  MN21  where  Is  bast  single  level  choice. 
If  MN2jo  ‘  MN2i  £.  M»  proceed  to  level  3.  If  MN2j0  -  MN2i  <  M, 
aijo  *8  chosen.  (In  other  words,  we  were  examining  to  find 

at.  Let  at  -  SjLjo  .) 

A.  Level  3:  Consider  two  cases:  a^j0  wins  and  a^j0  loses. 
Using  the  single  level  algorithm,  find  the  N  best  tuples  for  each 
case.  Proceed  in  each  case  as  in  level  2.  Use  the  case  with  the 

j 

least  potential  yield  for  the  test  with  parameter  M. 

5.  Continue  until  process  stops. 

Notes:  (1)  If  process  does  not  stop  until  all  outcomes  are 
determined,  it  would  be  a  branch  and  bound  technique.  (2)  The 
significance  of  parameter  N  is  that  N  branches  of  the  decision  tree 
will  be  investigated  at  each  level.  (3)  The  significance  of 
parameter  M  is  that  if  the  level  J  search  does  not  yield  H  more 
outcomes  than  the  level  j-1  search,  then  no  further  levels  ire 
investigated. 
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SECTION  VI 


OTHER  APPROACHES 

1.  Let  S  be  the  full  set  of  n-tuples  that  define  our 
problem.  S  is  a  partially  ordered  set  (POS)  with  the  order 
relation  »  as  defined  in  section  III,  p.  12.  A  subset  C  of  S 
is  called  a  chain  if  x,y  in  C  implies  either  x  «  y  or  x  »  y. 

A  decomposition  of  S  is  a  family  (C|,  .  .  .  ,Ct)  of  disjoint 
chains  whose  union  is  all  of  S.  Such  a  decomposition  is  called 
minimal  if,  for  any  other  decomposition  {K^,  .  .  .  ,1^}  ,  t  <  r. 

There  exist  various  methods  for  finding  a  minimal  decomposition 
for  a  POS,  and  these  could  be  applied  before  any  experiments  are 
made  on  the  system.  (See  Ivanescu  [2].)  If  thore  are  dominance 
relations,  it  is  possible  that  the  POS  would  decompose  into 
several  long  chains.  (Example:  If  position  i  dominates  i+1 
for  all  i,  then  there  would  be  only  onf  long  chain1.)  Restricting 
our  attention  to  such  chains  would  allow  us  to  "linearise"  the 
problem  and  could  greatly  simplify  the  investigation. 

2.  A  geometric  approach  can  be  formulated  by  replacing 
program  Q  with  program  G,  described  below.  Consider  the  set  of 
all  undetermined  tuples.  (We  are  looking  at  table  T^.^.)  These 
are  points  on  a  lattice  in  n-space.  G  chooses  for  a^  one  of  the 
tuples  closest  to  c,  the  center  of  mass  of  these  points.  Intuitively 
c  has  the  property  that  any  n-1  dimensional  subspace  (a  plane,  if 

n  ■  3)  through  c  has  about  the  same  number  of  points  on  one  side 


as  the  other.  It  is  not  hard  to  bog  that  for  any  tuple  x  there 
exists  an  n-1  dimensional  subspnea  through  x  such  that  the  tuples 
determined  if  N(x)  ■  1  are  on  one  side  and  those  determined  if 
N(x)  *  0  are  on  the  other.  Since  we  wish  to  choose  in  a  way 
that  tends  to  minimise  |  #i.i<S|)  -  *  the  points 

closest  to  c  might  be  good  candidates  for  optimal  choices. 

A  few  cases  testing  G  hav<  been  computed ,  and  these  indicate 
that  G  is  often  close  to  Q  but  sometimes  falls  short.  Extensive 
tests  have  not  been  made. 

PROBABILISTIC  EXTENSION  OF  THE  GENERAL  PROBLEM 

It  has  been  suggested  that  it  would  be  worthwhile  to  extend 
tho  general  problem  in  the  following  way:  associate  with  each 
tuple  x  a  priori  probabilities  Pq(x)  ,  p^(x) ,  .  ,  .  ,plc(x) 
where  p^(x)  is  the  probability  that  N(x)  »  i.  Then  it  would 
be  necessary  to  defino  a  program  Qp  to  replace  Q  so  that  optimal 
choices  are  mede  using  the  probability  data.  We  are  currently 
working  on  thio  aspect  of  the  problem. 
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APPENDIX 

SAMPLE  SMEDAL  OUTPUT  FOR  A  SIMPLE  CASE, 

Note]  N-sequance ,  referred  to  on  the  next  page,  ie  the 
sequence  of  outcomes  datermlned  by  experiment. 

Also,  the  efficiency  of  the  algorithm  is  considerably 
improved  for  larger  problems,  (See  the  Table  in  Section  2) 
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132  HAS  B£6N  *U*i  VITH  DUTCO'SE 
AEXT  VUX  SNCUIC  3E _ 312 


DODGE  AWARDED  THE  1971  SAMUEL  S.  WILKS  MEMORIAL  MEDAL 


Tha  Praaantatlon  of  the  Savantaanth  Samuel  S,  Wilka  Award 
Made  by  Frank  E.  Grubbe 


Tha  Samuel  S.  Wilks  Memorial  Medal  Award,  initiated  in  1964  by  the . 

U.  S.  Army  and  American  Statistical  Association  jointly,  ia  administered 
by  tha  American  Statlatical  Association,  a  non-profit,  educational  and 
scientific  society  foundad  in  1839.  Tha  Wilke  Award  is  given  each  year  to 
a  statistician  and  ia  based  primarily  on  hla  contributions  to  the  advancement 
of  scientific  or  technical  knowledge  in  Army  statistics,  ingenious 
application  of  such  knowledge,  or  aucceeaful  activity  in  the  fostering  of 
cooperative  scientific  matters  which  coincidentally  benefit  the  Army, 
the  Department  of  Defense,  the  U.  S.  Government,  and  our  country  generally. 

The  Award  consists  of  a  medal,  with  a  profile  of  Professor  Wilks 
and  the  name  of  the  Award  on  one  side,  the  seal  of  the  American  Statistical 
Association  and  name  of  the  recipient  on  the  reverse,  and  a  citation  and 
honorarium  related  to  the  magnitude  of  the  Award  funds.  The  annual 
Army  Design  of  Experiments  Conferences,  at  which  the  Award  is  given  each 
year,  are  sponsored  by  the  Army  Mathematics  Steering  Committee  on  behalf 
of  tha  Office  of  the  Chief  of  Research  and  Development,  Department  of 
the  Army. 

The  funds  for  the  S,  S.  Wilks  Memorial  Award  were  donated  by  Philip 
G,  Rust,  retired  industrialist,  Thomaaville,  Georgia. 

Previous  recipients  of  the  Samuel  S.  Wilks  Memorial  Medal  include 
John  W.  Tukay  of  Princeton  University  (1965),  Major  General  Leslie  E, 

Simon (1966),  William  G,  Cochran  of  Harvard  University  (1967),  Jersy 
Neyman  of  rhe  University  of  California  (1968),  Jack  Youden  (1969) 
retired  from  the  National  Bureau  of  Standards,  and  George  W.  Snedecor 
(1970)  retired  from  Iowa  State  University, 

With  the  approval  of  President  Churchill  Eisenhart  of  the  American 
Statistical  Association,  the  Wilks  Memorial  Medal  Committee  for  1971 
consisted  of  the  following; 

Profesaor  Robert  E.  Bechhofer  -  Cornell  University 

Professor  William  G.  Cochran  -  Harvard  University 

Cr,  Fred  Frishman  -  Army  Research  Of fice-Washington,  D.C, 

Dr.  Badrig  Kurkjian  -  US  Army  Materiel  Command,  Washlngton,DC. 

Dr.  William  R.  Pabst,  Jr.  -  Washington,  D.  C. 

Major  General  Leslie  E.  Simon  -  Retired 

Dr.  John  W.  Tukey  -  Princenn  University 

Dr.  Frank  E,  Grubbs,  Chairman  -  US  Army  Aberdeen  Research  and  Development 

Center 

Aberdeen  Proving  Ground,  Maryland 


Preceding  page  blank 
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As  many  conferees  aca  aware,  our  process  of  saleeting  the  Wilks 
Medalist  each  year  turns  out  to  bi  a  statistically  significant  event, 
you  know,  screening  some  25-30  nominees. 

The  1971  Wilks  Memorial  Medalist  is  an  international  authority  and  a 
pioneer  in  developing  sampling  inspection  plans  and  quality  rating  methods, 

Ha  was  born  in  Lowell,  Massachusetts  in  1893  and  received  an  S.  B, 
Degree  in  Electrical  Engineering  from  the  Massachusetts  Institute  of 
Technology  in  1916.  After  teaching  Electrical  Engineering  for  a  year  at 
MIT  he  Joined  the  Western  Electric  Company  as  a  development  engineer  in 
the  Engineering  Department.  In  1924  ha  transferred  to  the  newly-formed 
Inspection  Engineering  Department  which  became  part  of  Bell  Telephone 
Laboratories  the  next  year.  In  that  department  were  W,  A,  Shewhart, 

G,  D.  Edwards  and  later  H,  G,  Romig  and  P,  S,  Olmstead  whose  names  ate 
also  wall-known  in  the  quality  control  and  statistical  fields. 

Our  1971  Wilks  Medalist  is  a  Fellow,  A  Shewhart  Medalist  and  an 
Honorary  Member  of  the  American  Society  for  Quality  Control,  A  Fellow 
of  the  American  Statistical  Association  and  a  Fellow  of  the  Institute  of 
Mathematical  Statistics.  From  the  American  Society  of  Testing  and 
Materials  ha  has  received  the  Award  of  Merit  and  an  honorary  membership. 

Ha  is  internationally  known  for  hie  published  articles  on  sampling 
inspection  plans  and  the  book  Sampling  Inspection  Tables  which  was  Jointly 
authored  with  Harry  Romig.  In  addition  to  the  single  and  double  nampling 
plans  presented  in  the  book,  he  has  published  many  special  purpoms  plans. 
They  Include  continuous  sampling  plans  for  conveyorlsad  production, 
chain  sampling  plans  which  may  be  used  for  characteristics  requiring 
destructive  or  costly  tests,  skip-lot  sampling  plans  applicable  to 
chemical  physical  analyses  of  raw  materials  and  a  cumulative  results  plan 
which  la  superimposed  on  regular  acceptance  sampling  plans  having  small 
sample  sizes. 

Well  known  also  are  his  contributions  to  the  development  of  the 
Army  Ordnance  Standard  Sampling  Inspection  Tables  which  were  used  during 
World  War  II  for  the  inspection  of  materiel  and  his  contributions  to 
the  well-known  MIL- STD-1 05  (especially  105D  issued  in  1963), 

Our  '71  Wilks  Medalist  also  contributed  significantly  to  the 
check  inspection  and  demerit  rating  plans  for  products  used  by  the  Bell 
System  and  prepared  a  quarterly  Quality  Report  that  showed  quality  rates  for 
important  products.  This  system  is  still  the  basis  for  judging  the 
quality  of  products  entering  the  communications  network. 

Throughout  his  career  he  was  also  a  teacher:  first  in  departmental 
courses  and  Out-of-Hours  courses  at  BTL,  then  in  the  Communications 
Development  Training  (Kelly  College)  program  for  new  BTL  engineers. 

After  hie  retirement  from  the  Laboratories  he  became  a  Professor  in  the 
Graduate  School  at  Rutgers,  The  State  University  of  New  Jersey. 
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Even  hie  work  in  preparing  manuals  for  ASTM,  the  Z  standards  for  the 
American  Standards  Association  and  definitions  in  the  Standards  Committee  of 
ASQC  were  a  form  of  teaching.  Many  of  his  teaching  techniques  were  also 
illustrated  in  his  Edward  Marburg  Lecture  (1954  Meeting  of  ASTM)  entitled 
"Interpretation  of  Engineering  Datat  Soma  Observations." 

It  has  been  obvious  that  for  soma  minutes  now  the  1971  Samuel  S. 

Wilks  Memorial  Medalist  is  Harold  F,  Dodge, 


THE  FOLLOWING  ACCEPTANCE  REMARKS  WERE  PREPARED  BY  HAROLD  F,  DODGE 
THEY  WERE  READ  AND  DISCUSSED  BY  LESLIE  E.  SIMON 


Mr,  Chairman  and  Members  of  the  Conference: 

This  is,  for  me,  a  very  happy  occasion  as  1  express  my  extreme 
pleasure  in  accepting  the  1971  Samuel  S,  Wilks  Memorial  Medal  which 
honors  our  long-time  friend  Sam  Wilks  and  the  extensive  contributions  he 
made  toward  the  applications  of  mathematical  statistics  to  problems 
of  national  interest.  Those  who  knew  him  well  will  alwaya  be  grateful 
for  his  kindness  and  helpfulness  in  translating  some  of  our  individual 
problems  into  the  language  of  mathematical  statistics.  I  well  recall 
that  when  he  looked  over  an  early  manuscript  on  the  continuous  sampling 
plan  in  19^0,  he  asked  nicely:  "Did  you  know  that  this  type  of  series 
is  called  a  'power  series' 7",  for  he  could  have  put  the  question  in  the 
more  provocative  form:  "Don't  you  know  tWat  this  type  of  series  is 
called  a  'power  series'?"  It  is  sometimes  the  little  things  that  you 
never  forget. 

World  War  II  brought  out  the  need  and  opportunities  for  broad 
Intensive  application  of  quality  control  and  statistical  methods. 

Progress  in  statistical  quality  control  (SQC)  had  already  gone  fairly 
far  in  the  Bell  System  since  its  initiation  in  1924.  There  it  was 
definitely  a  team  activity  with  close  cooperation  between  the  Quality 
Assurance  Department  of  the  Bell  Laboratories  (initially  called 
Inspection  engineering)  and  engineers  of  the  Installation  and  Manufac¬ 
turing  organisations  of  the  Western  Electric  Company.  In  1924, 

Shewhart  Invented  the  control  chart.  This,  together  with  the  develop¬ 
ment  of  single  sampling  and  double  sampling  inspection  plans  and 
quality  rating  methods,  led,  in  the  late  1920'e  and  during  the  1930's, 
to  the  quite  general  use  of  these  quality  control  methods  by  the 
installation  and  manufacturing  groups  of  Western  Electric,  tfa  all 
began  to  talk  of  3-sigma  limits,  assignable  causes,  classification 
of  defects,  single  and  double  sampling,  and  LTPD  and  AOQL  sampling 
tables.  And  in  1941  the  ao-called  Dodge-Romig  sampling  inspection  tables 
were  published.  Of  special  note,  too,  waa  the  related  work  dona  by 
Leslie  Simon  In  Picatinny  Arsenal  atartlng  in  1934.  There  In  the 
manufacturing  department  he  successfully  Introduced  quality  control 
techniques  Including  the  uae  of  control  charts  for  controlling  the 
weight  and  uniformity  of  powder  In  various  types  of  ammunition. 
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In  December  1940,  at  the  request  of  the  War  Department,  the  Amer¬ 
ican  Standards  Association,  using  its  emergency  procedure  appointed 
an  Emergency  Defense  Committee  Zl,  on  Quality  Control,  with  the  speaker 
as  chairman  and  with  Leslie  Simon  and  Ed  (W.  Edwards)  Darning  aa  two 
of  its  other  five  members.  The  first  two  standards^  Zl.l  and  El. 2,  on 
control  chart  principles  and  techniques  wsre  published  in  May  1941,  and 
the  third,  Z1.3  on  the  Control  Chart  Method  of  'Controlling  Quality 
During  Production  in  April  1942,  These  standards,  updated,  are  in  force 
todey. 

But  the  Ordnance  Department  of  the  Army  also  wanted  active  use 
of  quality  control  in  the  inspection  of  ammunition  and  weapons,  and  as 
a  result  of  discussions  with  Bell  Labs  Quality  Assurance  Department,  a 
new  Quality  Control  Section  was  created  in  the  Inspection  Branch 
(later  transferred  to  the  Production  Service  Branch)  In  Washington  in  the 
spring  of  1942,  headed  by  George  D,  Edwards,  our  Director  of  Quslity 
Assurance,  with  G.  Rupert  Cause  of  Ordnance  and  myself  as  active  partici¬ 
pants.  We  davalopnd  a  system  of  acceptance  quality  control  with  new 
standard  inspection  procedures,  and  with  assistance  at  the  Labs,  a  new 
set  of  sampling  tables  -  all  of  which  were  made  available  for  prompt  use 
in  an  Ordnance-wide  quality  control  training  program. 

The  sampling  tables  were  based  on  the  new  concept  of  an  Acceptable 
Quality  Level  (AQL),  and  three  kinds  of  inspection,  Normal  Tightened  and 
Reduced,  with  rules  for  switching  between  them,  as,  for  example,  for 
switching  to  a  more  exacting  or  Tightened  sampling  plan  when  quality 
runs  worse  than  the  AQL.  At  the  outset  ws  were  literally  scared  before 
making  the  first  presentation.  We  were  going  to  talk  about  defectives 
and  AQL's  that  implied  less  than  full  conformance  with  specifications. 

And  we  had  been  hearing  about  directives  that  in  effect  said  such  things 
ast  a  contractor  "will  not  ship  defective  material."  But  after  the 
first  session  with  demonstrations  of  sampling  with  a  bowl  of  chips  and 
a  quincunx,  things  seeir~  \  to  go  well. 

The  first  QC  training  courses  were  given  in  15  locations  starting 
in  Washington  in  August  and  winding  up  in  Salt  Lake  City  in  November,  1942, 
Each  conference  lasted  three  days,  with  lectures,  problem  periods  and 
homework.  During  this  period  we  three  learned  a  great  deal  about  the 
cooperation  and  dedication  of  our  civilian  and  military  personnel  in 
wartime,  and  I  might  add,  about  the  joys  of  night-time  travel  in  an  upper 
berth.  The  conferences  ware  attended  by  qualified  nominees  from  over 
80  Ordnance  establishments.  During  these  conferences  and  with  the 
rapidly  expanding  applications,  situations  arose  that  indicated  the  need 
for  extending  the  tablas  in  a  number  of  ways.  Larger  lot  sizes  were 
needed,  and  discussions  with  the  Signal  Corps  indicated  that  a  modified 
approach  was  desirable  for  complex  products  like  radio  equipments  made 
in  lots  of  10,  20  or  perhaps  100  each.  Telephone  experience  had  taught 
us  that  "defects-per-unit"  was  both  theoretically  and  practically  better 
than  "percent  defective"  as  a  measure  of  quality  of  complex  products. 

Work  on  a  complete  set  of  defects-per-unit  single  end  double  sampling 
tables  was  started  and  early  drafts  were  made  available  in  1943  to  Ordnance 
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aud  the  Signal  Corps.  These  new  tables,  designed  to  fit  in  statistically 
with  the  earlier  percent  defective  tables  to  fora  a  smooth  series  over 
the  complete  range  of  lot  sizes,  were  used  in  a  second  series  of  16 
training  conferences  that  we  conducted  under  the  auspice*  of  the  Army 
Service  Forces  (A.S.F.)  through  May,  June  and  early  July  1944,  This  time 
about  half  the  trainees  wets  from  Ordnance,  the  rest  distributed  among  the 
other  eight  departments  or  corps  of  the  A.S.F.  with  a  few  from  the  Navy 
and  Air  Force.  The  complete  set  of  12  Single  and  Double  Sampling  Tables 
was  widely  used  through  the  rest  of  the  war  and  served  ae  a  reference 
standard  for  sampling  by  the  technical  services  of  the  military.  New 
Navy  tables  in  May  194S,  using  closely  the  same  procedures,  included 
sequential  or  multiple  sampling  plans  as  veil  as  single  and  double. 

As  reported  by  Sem  Wilks  in  a  1947  paper,  a  nation-wide  program 
of  short  intensive  quality  control  courses  was  sponsored  in  1943  by 
the  Office  of  Production  Research  and  Development  (OPRD),  Thirty-four 
so-called  "8-day  wonder  courses"  were  given  in  nearly  25  cities  to  a  total 
of  about  2000  individuals  representing  800  industrial  organizations. 

The  texts  used  included  tha  Z.L  A.  S.  A.  Standards  on  Quality  Control  and 
the  Army  Ordnance  and  A.S.F.  sampling  tables. 

After  the  war,  a  Joint  committee  worked  out  a  new  military  standard 
(105A)  with  sampling  tables  that  were  a  compromise  between  the  Army 
and  Navy  wartime  tables.  The  current  MIL-STD-105D  is  a  1963  revision 
-  the  result  of  international  cooperation  by  an  ABC  (America,  Britain, 

Canada)  working  group.  It  haa  many  of  the  basic  original  features, 
such  as  AQL,  Normal,  Tightened  and  Reduced  Inspection,  but  incorpor¬ 
ates  a  number  of  changes  including  sosm  I  suggested  in  a  1959  Rutgers 
Technical  Report  -  one  of  which  was  a  much  simplified  rule  of 
switching  between  Normal  and  Tightened  Inspection.  MIL-STD-105D 
la  still  widely  used  here  and  abroad. 

Now  there  are  situations  where  it  is  neither  convenient  nor  practical, 
as  in  conveyorlsed  production,  collect  product  units  in  lots.  This 
led  to  the  development  in  1940  of  the  continuous  sampling  plan,  CSP-1, 
which  usee  alternate  periods  of  sampling  and  100X  inspection.  This  plan 
was  made  available  to  the  Western  Electric  Company  and  following 
successful  uae  it  was  published  in  1943.  A.  C.  Cohen  tells  of  its  applicatlor 
with  "considerable  success"  at  Picatinny  Arsenal  in  late  1943,  citing  an 
examp) a  o i  controlling  weights  of  bag-loaded  charges  of  smokeless 
powder , 

A  number  of  epeciel  purpose  eampling  plans  have  also  been  made 
available.  Among  these  ere  the  "chain"  eampling  plane  in  which  the 
acceptance  or  rejection  of  e  lot  is  basad  on  the  combined  sampling 
results  for  the  current  let  end  one  or  more  preceding  lots,  with 
cumulation  starting  afresh  after  each  lot  rejection.  Chain  sampling 
makes  possible  e  sizable  reduction  in  temple  size. 
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Skip-lot  sampling  plana  hava  baan  developed  to  provlda  a  diffarant  Z 

but  almplar  fora  of  reduced  inapection,  by  aklpping  the  inapactlon  of  a  j 

given  proportion  of  lota  vhan  quality  ia  good.  The  atructure  of  these  1  j 

plana  is  baaed  on  the  theory  of  the  continuoua  sampling  plan.  CSP-1,  already 
mentioned. 

A  cumulative-results  plan  uaing  a  cumulative  results  criterion  3 

(CRC) ,  published  by  A.  F.  Cons  and  myself  in  1964  provides  a  special 
process  correction  feature  applicable  especially  when  lots  and  samples 
are  smell.  Experience  with  its  use  in  quality  assurance  inspections 
by  A.E.C.  Inspection  agencies  suggest  that  the  moat  promising  solution 
for  the  smell  sample  problem  is  enforced  correction  of  the  process  through 
the  use  of  collective  hietory  and  a  cumulatlva-results-criterion  for 
action. 

Two  special  projects  coverod  by  Army  contracts  with  tha  Quality  1 

Assurance  Department  of  Bell  Labs  may  be  mentioned.  The  first  in 
1952  related  to  aettlng  up  Quality  Control  Procedures  (QCP's)  for 
controlling  the  ballistic  quality  of  105mm  howltaer  ammunition  from 
a  single  loading  line.  In  this  project  at  Joliet  Arsenal,  the  goal 
was  larga  lots  of  uniform  quality  and  all  were  pleased  with  the 
successful  production  of  an  initial  large  lot  of  over  149.000  rounds. 

The  second  project  was  the  development  in  1955-56  of  an  over-all 
quality  assurance  plan  for  the  NIKE  I  round,  with  a  total  of  38 
Quality  Assurance  Procedures  (QAP'a)  covering  inspections,  tests, 
quality  surveys  and  quality  rating.  The  intent  of  this  plan  was  to 
provide  Ordnance  with  a  continuing  basis  for  satisfying  itBelf  that 
the  quality  of  complete  rounds  was  what  it  should  be.  With  relatively 
small  quantities,  continuous  sampling  inspection  plans  were  used. 

Here  again  we  learned  some  new  terminology,  as  for  example,  when  we 
asked  what  that  little  motor  vehicle  was,  like  an  open  vat  on  wheels 
crawling  across  an  open  green,  and  were  told  it  was  toting  "nltro" 
and  was  known  as  an  "angel-buggy." 

In  conclusion,  as  Sam  Wilks  said  in  commenting  on  the  use  of  SQC 
during  the  wart  "One  would  have  to  conclude  that  it  actually  took  a 
war  to  demonstrate  that  the  methods,  when  applied  on  a  wide  scale, 
constitute  an  important  national  resource." 


REMARKS  ON  THE  ACCEPTANCE  OF  THE  S.  S.  WILKS  MEDAL 
FOR  HAROLD  F.  DODGE  BY  LESLIE  E.  SIMON 


The  work  of  Harold  F.  Dodge  speaks  for  itself  as  to  ita  importance 
and  usefulness  to  the  Army  and  to  the  world  in  general.  However,  as  one 
of  his  many  old  friends  and  colleagues,  I  would  like  to  eay  a  few  brief 
words  about  his  qualities  as  a  man.  Those  are  qualities  that  have 
endeared  him  to  us  and  have  also  affected  his  work. 
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Among  the  times  that  I  had  opportunity  to  obaerva  him  carefully,  was 
whan  ha  was  the  chairman,  and  1  a  member  of  the  Emergency  Defer se  Committee 
on  Quality  Control,  Ve  ware  writing  the  American  War  Standarda,  Zl. 1 
Z1-.2  and  Z1.3,  to  which  Dodge  has  referred.  The  work  was  not  only  crucially 
important,  but  was  also  dona  under  pressure  because  they  were  needed 
for  Immediate  use.  Dodge's  personal  characteristics  were  in  -large 
measure  responsible  for  the  metlcuioua  correctness  of  the  work,  which 
enhanced  the  respect  and  confidence  in  which  they  were  held,  and 
thereby  promoted  their  prompt  and  wide-spread  use.  In  the  meetings  of 
the  committee,  no  question  asked  him  or  suggestion  made  to  him  was  ever 
so  simple  that  it  did  not  get  a  courteous  answer,  nor  so  complex  that 
he  summarily  dismissed  it  from  discussion,  I  often  marveled  at  hie 
patience,  consideration  and  generosity. 

He  is  one  of  the  most  thorough  workmen  that  I  have  ever  known.  It 
would  seem  to  me  that  a  task  was  completed,  but  Harold  would  look  at 
it  critically  from  every  possible  angle,  and  often  bring  to  light 
conditions  that  were  valid  to  the  problem  and  had  not  been  covered. 

When  he  felt  that  an  assignment  had  been  completed,  it  was;  and  only  new 
conditions  or  newly  discovered  knowledge  was  likely  to  necessitate  its 
revision. 

It  is  not  only  his  competence,  depth  of  knowledge,  and 
willingness  to  help  any  worthy  person  who  asks  hiB  guidance  that  keeps 
him  busy  in  doing  gratuitous  work,  but  his  warm,  out-going  friendliness 
that  makes  him  spontaneously  say,  "yes";  a  quality  that  endeares  him  to 
all  who  know  him. 


COMPARISON  OF  TREATMENTS  GIVEN  BI-VARIATE  TIME  RESPONSE  DATA 


Pearl  A.  Van  Natta 

U.  S.  Army  Madlcal  Research  and  Nutrition  Laboratory 
. . Denver,  Colorado . 


This  papar  ia  concerned  with  tha  question  of  "How  may  traatmanta  ba 
'bast*  comparad  givan  bi-variata  time  rasponaa  data?"  Tha  experimenter's 
plan  for  obtaining  tha  data  haa  had  prallminary  taating.  A  quaation 
which  I  would  like  tha  panal  to  kaap  in  mind  during  tha  axpoaltion  la, 

"Can  thia  data  laad  to  anothar  plan  which  would  allow  mora  meaningful 
comparison*?" 

The  following  abstract  of  tha  experimental  situation  will  give 
a  background  for  tha  specific  questions  which  require  illumination, 
if  not  answers,  by  statistical  analysis. 

Baby  chicks  were  used  in  studies  designed  to  determine  whether  or 
not  any  of  the  compounds,  histidine,  histamine,  and  aspirin,  would 
correct  tha  abnormality  of  aulfate  metabolism  in  bone-fomslcg 
regions  associated  with  sine  deficiency,  since  thay  correct  the  gross 
leg  abnormalities  similarly  associated  with  a  deficiency  of  the  trace 
element.  These  symptoms  era  similar  to  symptoms  of  rheumatoid  arthritis 
in  humans.  The  three  compounds  were  supplied  in  tha  diet  in  the  following 
concentrations t  hiatidlne,  1.0X,  histamine,  ,2Z,  and  aspirin,  ,75X. 

A  basal  fsad  supplied  a  standard  amount  of  aulfur.  Following  4  or  5 
weeks  of  these  traatmanta  on  either  a  Zn-deficiant  (14ppm)  or  a  Zn- 
adequate  (94  ppm)  diet,  the  birds  were  orally  dosed  with  10  pC.  of 

35  35  1 

Ns j  SO^,  The  S  content  of  the  epiphyseal  plate  (EF)  and  the  primary 

spongiosa  (PI,  from  the  tibia  was  assayed  at  either  6,  12,  or  24  hours 
following  isotope  dosage.  There  wars  24  random  samples  of  chicks  with 
sample  siaes  ranging  from  10  to  18,  ([4  drugs]  x  [2  Zn  levels]  x 

[3  kill-times j ) ,  Tha  data  are  radioactive  counts  from  a  liquid 
scintillation  counter  which  are  adjusted  for  background  and  efficiency  of 
tha  machine,  divided  by  the  actual  milligrams  of  tissue  counted,  and 
called  dpm/mg  tissue.  The  exparl"”  itsl  tissue  was  obtained  from  tha 
end  of  the  tibia,  Tha  EF  caps  the  bone  and  haa  a  different  texture 
than  the  surrounding  tissue  so  that  the  experimenter  has  confidence 
that  the  tissue  counted  is  actually  EP,  The  PS  lias  directly  under 
the  EP  and  haa  a  similar  texture  to  the  mineralised  tissue  beneath  it. 

It  was  more  difficult  to  isolate.  The  standardisation  cf  tha  counts 
was  necessary  because  one  of  the  manifestations  of  sine  deficiency  is 
a  slow  growth  rata  so  that  soma  groups  consisted  of  animals  with  much 
smaller  tlbiaa  and  hence  smaller  amounts  of  BP  and  PS  than  other  groupa. 

An  unknown  quantity  in  this  experiment  is  the  relationship  of  the 
countable  sulfur  to  the  total  aulfur  in  the  tissue.  Another  unknown 


quantity  ia  what her  the  radioactive  aulfur  difference*  between  treatment* 
are  due  to  changed  uptake,  changed  excretion  or  both.  These  could  ba 
Incorporated  in  future  deaigna.  The  knowledge  obtained  can  be  useful 
in  underetanding  wound  healing  and  fracture  healing. 

The  experimenter's  questions  and  the  results  of  preliminary 
statistical  analysis  (a  ■  .05)  are: 

1.  Does  the  linc-dif lclant-only  diet  stimulate  a  different  sulfur 
response  from  the  a lnc-adequate-only  diet?  (Figure  1). 

The  preliminary  data  when  analysed  in  a  univariate  way  for  each 
kill-time,  using  the  8teel-Dwass  2-sample  rank  statistic,  show  that 
the  locations  of  the  EP  values  of  the  two  treatments  differ  at  6  and  12 
hours  but  not  at  24  hours  and  that  the  location  of  the  PS  values  differ 
at  12  and  24  hours  but  not  at  6  houre.  The  direction  of  the  difference 
Is  that  the  Zn-adequate  animals  have  more  countable  sulfur  in  the  EP  tissue 
at  6  and  12  hours  but  not  at  24  hours  and  more  in  the  PS  tissue  at  12 
and  24  hours  but  not  at  6  hours. 

Figure  1  shows  that  ths  passags  of  time  produces  changes  in 
variation  in  the  EP  and  PS  that  is  more  pronounced  in  the  Zn-adequata 
groups  than  in  the  Zn-deficient  groups.  The  requirement  for  confldancs 
intervals  for  the  amount  of  the  translation  between  the  two  populations 
is  that  the  only  possible  difference  between  ths  distributions  tasted  Is  a 
translocation.  Hence,  these  were  not  computed. 

2.  Do  the  Zn-deficient  +  (a  drug)  diets  produce  reaponses  different 
from  the  Zn-deficient-only  diet?  (Figure  2). 

The  preliminary  data  when  analysed  in  a  univariate  way  for  each 
kill-time  using  the  Steel  many-one  rank  statistic  gave  the  information 
that: 

a.  Aspirin  stimulates  more  aulfur  incorporation  in  both  EP 
(at  6  and  12  hours)  and  PS  (at  12  hours); 

b.  Histamine  stimulstse  more  aulfur  incorporation  in  both  EP 
(at  6  and  12  hours)  and  PS  (all  times); 

c.  Histidine  stimulates  more  sulfur  incorporation  in  EP  (at  12 
hours)  and  PS  (at  12  and  24  hours). 

3.  Do  the  Zn-adequate  +  (a  drug)  diets  produce  responses  different  from 
the  Zn-adequate  only  diet?  (Figure  3). 

The  univariate  Steel  many-one  rank  statistic  gave  the  information 


a.  Aspirin  stimulates  nor*  sulfur  Incorporation  in  EP  (all  tlisas) 
but  not  in  PS  (no  tins). 

b.  Histamine  stimulates  more  sulfur  Incorporation  in  EP  (all  tlsies) 
and  PS  (24  hours). 

c.  Histidine  stimulates  more  sulfur  incorporation  at  EP  (6  hours) 

only. 

A  three-way  anova  (approximate)  on  aach  variable  separately  gave 
results  that  Zn-lavel,  kill-time,  and  drugs  all  have  significant  affects. 
Specifically,  for  EP  tissue  the  Zn-adequate  treatment  produces  more 
sulfur  than  the  Zn-deficient  treatment;  1?.  hours  shows  more  sulfur  than 
6  or  24;  and  the  treatments,  no  drug  and  histidine,  give  an  indistinguishable 
sulfur  response  while  aspirin  and  histamine  give  an  indistinguishable 
sulfur  response  that  is  higher  than  tha  response  elicited  by  the  other 
pair.  For  PS  tissue,  the  Zn-adequate  treatment  also  produces  more 
sulfur  than  the  Zn-daflclent  treatment;  24  hours  ehows  mors  sulfur  than 
A  or  12;  the  treatment  histamine  shows  greater  sulfur  response  than  Che 
other  treatments. 

4.  If  the  Zn-adequate  diet  produces  a  different  sulfur  response  from 
the  Zn-deficient  diet,  do  any  of  the  drugs,  when  added  to  the  Zn-deficient 
diet,  produce  a  pattern  end  level  like  the  Zn-adequate  diet?  (Figure  4.) 

Tha  one-tailed  Steel  many-one  rank  statistic  was  used  slncu  It 
was  of  interest  to  pick  out  the  drugs  which  allowed  less  incorporation 
of  sulfur  than  the  standard.  The  remaining  drugs,  whose  sulfur  affects 
were  the  same  or  greater  than  the  standard,  could  be  used  for  further 
experimentation.  The  information  is  that: 

a.  Aspirin  shows  no  different  effect  for  EP  tissue,  but  allows 
less  sulfur  to  be  incorporated  in  PS  (6,  12  and  24  hours). 

b.  Histamine  shows  no  different  effect  for  EP  tissue,  but  allows 
less  sulfur  in  PS  (12  hours). 

c.  Histidine  shows  no  different  effect  for  EP  tissue,  but  allows 
less  sulfur  in  PS  (6  and  12  hours). 

The  overall  impression  from  these  myriad  tests  is  that  histamine 
is  "best."  However,  litLlc  confidence  can  be  placed  In  it,  since  the 
whole  animal  has  been  lost  in  all  the  manipulations.  A  multivariate 
approach  might  be  an  entry, 

A  parametric  multivariate  approach  to  any  of  these  questions 
has  the  restriction  that  the  variance-covariance  matrices  from  the 
24  groups  cannot  be  considered  to  be  similar.  The  variation  in  both 
measurementa  varies  with  kill-time  and  drug  treatment,  A  t?*t 
which  does  not  require  homogeneous  variance-covariance  matrices  was 
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reported  by  Malle  and  Schultz  at  the  August  1971  meetings  of  the 
ASA  In  Fort  Collins,  The  test  Is  a  randomisation  test  and  little  la 
known  about  its  power.  It  Is  set  up  for  the  two-sample  problem  and 
makes  &  judgment  as  to  whether  the  two  groups  are  significantly  different 
or  not.  The  variables  should  be  scaled  by  ranking  the  data  within  each 
variable  or  standardizing  Che  data.  If  a  significanc  difference  is 
found,  then  the  k  measurements  can  be  examined  individually  to  determine 
where  the  differences  lie  without  affecting  the  error  rate. 

Would  this  multivariate  approach  provide  a  way  to  get  results 
for  question  47  Each  drug  group  could  be  tested  separately  with  the 
Zn-adequate  group  on  a  six  variable  basis.  But  then  how  to  decide  which 
is  best  if  there  are  competitors? 

The  problem  is  now  referred  to  the  panel  for  discussion. 


-370- 


PS,  HUNDREDS 


EUVAPIATF  SULFUR  COUNT  PLOT  I  f.U  GY  KILL- TIME  R  DfiUS  1  RE>TM£IJT 
Zll.'O  AD  CO  U  AT  t.  ONI  Y  C.  ZltlC  UF.TICIEI 'T  GNl.Y 


•  I A  ppm  71 NC 
*94  ppm  ZINC 


EP,  HUNDREDS 


FIG.  I 


-371 


B I  VAni'-.i 


run  count  i  ioiVfcU)  o v  mli.tilu:  l.  d:;ug  ikiatm cur 
,  zinc  dehch-nt  only  and  zinc  deficient  v.rrn  di;ugc 
v  i 

■  i 


X 

a*  C 


* 

*A. 
'A  1 


•f,A 


A  o 


ko  »*.{.•; 


o' 


5l 


ALL  14  ppm  ZINC 
•  ONLY 

■  histidine 

o  IIISTIMINC. 
aASMKIN 


IjJ 

a: 

Q 


3 

X 


4 

3 

2 


(O  , 

a 


o 


* 


k 

A 

A 


**  w“*  A 

*•<? 


O 


A 


12 


3- 

2' 

I- 

o-L 

o 


•  .* 


23  4  5  C  7  8  9  10  II 

E  P,  HUNDREDS 

FIG.  2 


-J  72- 


PS,  HUNDREDS 


bivariate  cui.ru:;  count  plot  run  by  kili-tii:?  n  drug  treat  went 
zinc  ad::oua‘i e  c:hy  and  zinc  adequate  with  drugs 


G- 

5 

4- 

3- 
2- 

1- 
0 

5- i 

4- 
3- 

2- 

1- 

0- 

3- 

2- 


A 


o 


tsnufjp. 


0 

M 


O 


O 


•V.  5 


,  % 


O 


o 


A 


A 


AA 


A 


24 


ALL  94  ppm  ZINC 

•  ONLY 

*  HISTIDINE 
o  HISTAMINE 
a  ASPIRIN 


X 

^  *  A 


A 


A  * 
A  x 
*  X 
A 


A 


A 


A 


IS 


o 


EP,  HUNDREDS 


FIG.  3 


6 


J 


373- 


PS,  HUNDREDS 


BIVAKiVTE  SULP'.'R  CUUMT  P  1.0 1  ifcO  CY  KILL*  TU'.L*  ft  D11U0  TflCATM!;! 
ZINC  ADEQUATE  ONLY  AND  ZINC  DEI-TCIENT  WITH  DRUGS 

"1  i  mu:; 


w  0  0 

*«  a  * 

•  W  t'/V  ,A 

*  O  #  £k  ©  M  A  •* 

o  #A 

©  N  A  ® 


•  94  ppm  7II4C  ONLY 

*  14  ppm  ZINC  WITH  HISTIDINE 

O  14  ppm  ZINC  WITH  HISTAMINE 
a  14  ppm  ZINC  WITH  ASPIRIN 


A  A 
X« 


•  •  »  - 

A  aCc?*,  c 
•  A!  i  ?  *  *A  X  %  * 


x  *  *A 

*0  ft 


fc  *  * 


4  5  6  7  8 

E  P,  HUNDREDS 


s  to 


-3/4- 


EXPERIMENTAL  DESIGN  IN  PROSPECTIVE  STUDIES  OF  INFECTION  IN  MAN 


Major  Robin  T.  Volltner 

US  Army  Medical  Research  Institute  of  Infectious  Diseases 
Fort  Detrick,  Frederick,  Maryland 


The  time  course  of  an  infectious  disease  in  man  is  poorly  under¬ 
stood,  Especially  little  is  known  about  the  infection's  incubation 
period,  when  no  symptoms  are  apparent.  For  this  reason  prospective 
studies  of  Infection  in  man  have  been  conducted  and  have  revealed 
changes  in  metabolism  during  and  after  thu  incubation  period.  Sandfly 
Fever,  a  self-limited  febrile  viral  illness  in  man,  has  been  the 
infection  model,  and  Army  volunteers  have  been  used  as  teBt  subjects. 

Several  biological  parameters  are  studied  throughout  the  course  of 
one  of  these  experiments.  For  any  one  parameter  there  are  the  three 
key  variables  of  test  subject,  infection,  and  time  which  together  or 
separately  Influence  that  parameter.  Furthermore,  the  relationship 
between  these  parameters  is  also  expected  to  be  important  to  under¬ 
standing  the  infectious  process.  With  these  thoughts  in  mind,  then,  the 
collected  data  has  been  analyzed  with  a  linear  statistical  model,  and 
some  preliminary  results  will  be  presented, 

The  experimental  design  can  be  outlined  as  follows.  First,  let 
us  denote  a  single  continuous  parameter  being  measured  during  the 
experiment  with  the  letter  "Y".  It  could  represent  the  concentration 
of  a  serum  substance,  for  example,  Now  we  indicate  two  subscripts  to 
the  variable:  i  to  represent  the  experimental  subject  and  j  to 
indicate  the  measurement  time  period.  Thus  the  experiment  with  respect 
to  one  measurable  parameter  is  represented  by  the  array  in  figure  1, 
where  time  periods  1,2, ...,S  comprise  the  baseline  time  interval,  the 
period  of  inoculation  with  the  infectious  material  being  S  and  where 
subjects  1,2,...,C  are  sham  inoculated  controls  while  subjects 
C+l , . . . ,N  are  truly  infected. 

The  questions  to  be  answered  by  this  prospective  study  of  infection 

are: 

1.  Are  there  significant  individual  differences  in  the  level  of  Y? 

2.  Does  Y  change  because  of  infection?  In  particular  does  Y 
change  prior  to  fever  development? 

3.  Do  the  artificial  constraints  of  the  experiment  (e.g.  diet, 
activity  level,  or  mental  status)  change  the  parameter  Y? 


-375- 


An  example  of  data  from  one  control  (marked  C)  and  two  infected 
(marked  I)  la  represented  in  table  1.  The  first  measurement  time 
period  is  to  the  left  of  each  line,  and  the  Inoculation  period  is  the 
sixth.  The  parameter  im  the  serum  concentration  of  the  amino  acid 
leucine  with  dimensions  uM/100  mi. 

The  next  figure  (figure  2)  is  a  graph  of  leucine  serum  concentration 
vs.  time  peri oda  ot'  ciia  experiment.  Each  point  on  the  curve  marked 
"Infected"  is  an  average  of  9  subjects'  measurements  v/hereas  the 
points  on  the  "control"  curve  are  averages  of  3  subjects.  Furthermore, 
the  two  points  on  the  "lnfectm&"  Curve  and  marked  with  standard  error 
brackets  deviated  slgni/iCMitlj'  (p  <  ,05)  from  preinfected  data  using 
a  paired  t  test  where  each  infected  subject  was  compared  to  his  own 
preinfected  average.  Such  t  test  analysis  has  been  the  routine  kind 
of  statistical  hypothesis  testing  performed  on  this  data,  although 
it  was  recognized  there  were  the  disadvantages  of  dependence  between 
time  periods  and  difficultly  specified  type  I  errors  associated  with 
the  t  test  for  this  data.  Thus  a  more  appropriate  method  of  analysis 
is  sought  such  that  it  uses  all  the  data  for  one  parameter  collected 
in  an  experiment,  and  we  have  considered  first  a  linear  statistical 
model  of  type  1  (see  reference  1). 

Referring  to  figure  3A  the  proposed  linear  model  partitions  the 
data  matrix  (Y,  )  into  five  different  regions  or  cases.  The  symbol 

a,  represents  the  effect  of  the  i  subject;  3,  represents  the  effect 

1  th  J 

of  the  experimental  conditions  otl  the  j  day;  and  Yj  represents  the 

effect  of  infection  on  day  j.  The  error  terms  are  assumed  to  be 

normally  and  independently  distributed  with  equal  variances  and  zero 
means.  Whereas  the  control  subjects  are  assumed  to  respond  only  to 
individual  effects  and  artificial  experimental  effects,  the  infected 
subjects  are  assumed  to  respond  further  to  the  a  efiect  of  infection. 

The  assumed  model  then  can  be  represented  by  the  matrix  equa  ion 


(Y1j)  -  X  B  +  («  ) 

where  (Y^),  X,  and  B  are  Indicated  in  figure  3B, 

We  used  a  version  of  the  BMD  General  Linear  Hypothesis  program 
(2)  and  obtained  the  estimates  in  figure  4.  The  test  of  the  hypothesis 
that  *  0,  i  «  1,2,...,N  yielded  an  F  significant  at  p  <  .01  so  that 
this  hypothesis  could  be  rejected.  However,  the  hypothesis  »  0 

j  -  2,.,,,T  and  Y^  ■  0,  j  *  S11,...,T  could  not  be  rejected.  It  appears 

that  this  result  was  obtained  because  the  linear  model,  unlike  the  t 
test  used,  considered  the  drop  in  leucine  level  that  occurred  both  In 
infected  subjects  and  control  subjects. 


This  is  the  extent  of  the  analysis  done  to  date,  but  we  currently 
plan  to  use  a  multivariate  approach  (since  many  parameters  are  measured 
during  an  experiment)  to  determine  correlation  between  parameters  and 
to  determine  whether  the  studied  parameters  can  discriminate  between 
infected  and  control  states. 
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TREATMENT  OF  NULL  RESPONSES 


Genevieve  L.  Meyer  and  Ronald  L,  Johnson 
U.S.  Army  Mobility  Equipment  Research  and  Development  Center 
Fort  Belvoir,  Virginia 


I.  INTRODUCTION. 


In  the  design  of  many  experiments  involving  equipm*-^  and  human 
observers  for  data  generation,  a  time  limit  or  dlstafK9  requirement  is 
often  present.  For  example,  lr.  an  air  obsarvation  study,  the  subject, 
serving  as  an  air  observer,  has  a  time  limitation  in  which  to  generate 
a  response.  When  the  subject  fails  to  respond  within  the  allowed  time 
interval,  his  action  la  classified  as  a  null  response. 

The  problem  to  be  considered  is  the  anlaysis  of  data  containing  null 
responses  where  the  null  responses  may  be  related  to  simulated  artifici¬ 
alities  and  the  sample  size  is  small.  Field  studies  are  usually  coatly  and 
the  availability  of  subjects,  in  most  cases,  is  limited,  making  re-runs 
prohibitive.  The  study  of  Johnson  (1971)  will  be  the  subject  of  this 
paper.  A  similar  study  la  that  of  Cavineas,  Maxey,  and  McPherson  (1971). 

U.  DESIGN  OF  EXPERIMENT 

The  object  of  the  Johnson  field  study  was  to  evaluate  the  camouflage 
effectiveness  of  the  two  competing  camouflage  items.  The  study  was 
c<  fii.vj.ctad  in  the  snow  fields  at  Bridgeport,  California.  Two  arctic-like 
sites  ware  selected  with  snow  depths  of  10  to  12  feet.  One  item  was 
placed  at  each  site.  The  items  were  interchanged  halfway  through  the 
test  aeries  to  prevent  teat  bias  due  to  site  peculiarities.  Sixteen  trained 
U.S.  Marine  air  observers  were  used  as  subjects,  They  were  flown  in  a 
Super  Sky  Chief  Cessna  337  with  all  observations  from  the  co-pilot's 
seat.  The  teBter  sat  directly  behind  the  subject  and  notified  the  subject 
of  the  start  and  termination  of  each  run.  Each  subject  wae  given  an 
opportunity  to  view  both  test  sites.  Half  of  the  subjects  were  flown 
over  the  one  site  first  and  half  over  the  other  site  first.  Two  altitudes 
were  used,  1300  feet  altitude  with  target  lateral  displacement  of  1000 
feat,  and  500  feet  in  altitude  with  target  lateral  displacement  of  650 
feet.  The  slant  ranges  were  1772  1032  feet,  respectively.  If  the 

subjects  detected  the  target  on  the  higher  altitude,  the  lower  flight  was 
cancelled  because  the  subjects  would  see  the  cam ovfi+mp  hen*  as  soon 
as  physically  possible  (site  familiarity).  The  tSx.tTSft  flew  on  a  heading 
which  minimized  the  effect  of  the  sun.  Each  subject  was  debriefed  at 
the  conclusion  of  the  test  in  order  to  obtain  complete  test  information 
concerning  their  responses.  Figure  1  represents  the  test  design. 


The  remainder  of  this  article  has  been  reproduced  photographically  from 
the  author's  manuscript. 
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III. 


RESULTS  OF  EXPERIMENT 


The  results  of  the  experiment  <uo  as  shown  in  Figure  2.  The 
response  was  the  slant  range-  in  ioot  at  which  the  detection  was  made. 

The  asterisk  reflects  a  null  : spoils  .1,  that  is,  no  response  was  made. 

The  few  responses  rocorded  moke  .noanintjful  analysis  difficult  -  if  not 
impossible. 

IV.  INTERPRETATION  AND  ANALYSIS  OF  RESULTS 

One  difficulty  in  the  analysis  of  the  results  is  the  treatment  of 
the  null  responses,  Presumably,  detections  would  have  occurred  at 
closer  rang  os .  Ono  approach  to  the;  problem  is  to  assign  a  default 
value  t.o  the  null  responses.  Posstblo  default  values  aro  1700  loot  for 
1300  foet  altitude  and  1000  feet  for  500  feel  altitude.  Those  default 
values  represent  the  minimum  slant  range  for  each  altitude,  i.o.r  the 
"last"  opportunity  the  subject  has  to  make  a  detection  butene  the  air¬ 
craft  proceeds  past  around  zero.  Those  values,  obviously,  overestimate 
the  detection  probability.  Yet,  using  zero  ranges  underestimates  the 
dote  ;t ion  probability.  Figures  3,  4  ,/fi,  and  6  show  pseudo  tost  results 
using  the  default  values  and  t1 .« .-Jr  rfhalysis  of  variance,  respectively. 

No  ■;! Mi'll  H  o.'n t  m,l  i*r;  ('ff(»«_-*te  nr  (j.rvl  '-nr.lnr  interactions’.  WCTO  fO’Jlld  • 

Another  approach  would  bo  to  use  the  available  data  to  extrapolate 
cumulative  detection  probability  vs.  lateral  range  and  ust;  the  resulting 
expected  detection  rang''  •  an  entries  in  the  preceding  test  results  matrix. 
The  results  should  then  bo  better  than  if  either  of.  the  extremal  values 
wero  used.  A  computer  program  called  MATCH  developed  by  1,.  Larson 
at  the  U.S.  Naval  Test  and  Evaluation  Detachment,  Key  West,  Florida, 
may  be  used  to  extrapolate  these  estimates ,  MATCH  accepts  a  minimum 
of  four  data  points  and  attempts  to  fit  those  data  to  both  n  normal  arid 
log-normal  distribution.  The  output  is  on  estimated  curve  depicting  Lhc 
cumulative  probability  of  detection  along  with  90%  confidence  intervals. 
The  original  data  In  cumulative  form  are  also  plotted  as  a  step  function. 
The  five  defections  for  item  two  were  run  through  the  MATCH  program  . 
Since  the  number  of  detections  wore  below  the  requited  minimum  it  was 
not  possible  to  use  '.he  MATCH  program  for  item  one.  The  results  are 
shown  in  Figures  7  and  fi. 

The  question  is  what,  if  anything,  can  be  sold  about  the  Improvement 
in  estimation  if  those  extrapolated  values  are  used?  Also,  what  effect 
would  the  use  of  those  extrapolated  values  have  in  the  analysis  of 
variance  results? 
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PSEUDO  TEST  RESULTS 
(In  root.) 

USING  1700  FEET  AND  1000  FEET  FOR  DEFAULT  VALUES 
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ANALYSIS  OF  VARIANCE  USING  MINIMAL  DEFAULT 
VALUES  OF  1700  I* LET  AND  lUOO  DM.'T 
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PSEUDO  TEST  RESULTS 
(In  !'eot) 
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ANALYSIS  OF  VARIANCE  USING  ZEROS  FOR  NO  RESPONSE 
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V.  CONCLUSIONS 


Difficulty  in  encountaj cd  in  ilio  analysis  of  the  test  data  m  which 
the  null  response  is  pro vn lent.  A  greatly  instnnscd  sample  size  Is 
required;  however,  this  ii‘  not  feasible  for  the  previously  slater)  reasons. 

VI.  OPEN  DISCUSSION 

Concepts  to  bo  considered  by  the  panelists  are  as  follows: 

n.  Improvement  of  the:  design  of  the  oxporimer'  ,  considering  the 
limited  sample  size. 

h.  Treatment  of  the  null  responses. 

c.  Analysts  of  the  test  data. 

d.  Recommendations. 
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COMMENTS  ON  THE  PAPER 


TREATMENT  OF  NULL  RESPONSES 
BY  GENEVIEVE  L.  MEYER  AND  RONALD  L.  JOHNSON 


James  J.  Filliban 
National  Bureau  of  Standard! 

Washington,  DC 

I  agree  with  the  authors  Meyer  ar.d  Johnson  that  some  fundaaental 
difficulties  do  exist  in  the  design  and  analysis  of  their  data,  vis., 
e,g.,  1)  the  duplication  and  hence  lack  of  independence  of  some  of  the 
observations;  2)  the  gather  small  sample  size;  and  3)  the  questionable 
fulfillment  of  the  ANOVA  assumptions  due  to  the  multiplicity  of  null 
responses. 

A  general  reminder  about  statistical  testing  is  that  for  a  given 
problem  and  a  given  sample  size,  there  may  exist  a  priori  no  possible 
outcome  that  would  be  statistically  significant  at  the  usual  .05  and  .01 
significance  levels.  A  simple  example  of  this  is  to  test  the  fairness 
of  a  coin  (i.e.,  test  Hq:  P  -  .5)  with  a  sample  size  of  n  »  3.  Even  if 
we  get  the  extreme  case  of  3  heads  (or  3  tails),  the  probability  (1/8  ■  12. 52) 
of  such  occurring  under  H  is  significant  at  neither  the  52  nor  the  17. 
level. 

A  closely  associated  point  to  be  remembered  is  that  for  a  given 
problem  and  a  given  sample  size,  there  may  exiBt  no  outcome  which  is  likely 
to  occur  which  would  be  significant.  For  example,  if  the  true  probability 
of  a  head  on  a  near-fair  coin  is  P  -  .51,  and  if  the  sample  size  is  11, 
then  the  probability  of  a  significant  event  at  the  52  level  (9  heads  and 
2  tails,  or  worse)  is  only  .04  and  so  there  is  but  1  chance  in  25  that  the 
bias  in  the  coin  will  be  detected. 

It  ia  my  feeling  that  the  camouflage  detection  experiment  of 
Meyer-Johnson  falls  into  this  latter  category;  that  is  to  say,  I  suspect 
that  the  detection  probabilities  are  so  low  and  so  close  to  one  another 
that  Che  .Likelihood  of  a  significant  outcome  (given  that  a  difference  in 
the  2  camouflages  does  exist)  is  extremely  low. 

If  I  properly  understand  the  problem  then  I  may  define  a  non- 
random  variable  P^  ■  Proh  (detection  using  camouflage  i),  where  i  ■  1,2. 

The  overtly  stated  objective  of  the  experiment  is  to  "evaluate  the  camou- 
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fl~ge  effectiveness  of  2  competing  camouflage  items,"  It  is  clear  that 
the  camouflage  affectlvanaaa  will  ba  (aa  the  authora  hava  taken  into 
account)  a  function  of  the  observer,  the  tine  of  day,  the  altitude  (or 
equivalently  the  alant  range  or  what  I  ahall  call  a imply  the  distance), 
and  the  camouflage  typea  (1  or  2).  Theaa  4  factora  are  not  of  equal 
importance.  I  believe  that  the  principal  problem  in  the  analyaia  ia  that, 
for  auch  a  small  sample  else  and  for  auch  apparent  a mall  differences  in 
detection  probabllltlea,  the  limited  amount  of  data  ia  being  aeked  too 
much  about  too  many  factora  and  ao  will  not  likely  ba  capable  of  yielding 
a  eignlflcant  reault  about  any  factor. 

It  la  further  clear  that  ia  really  a  non-random  function  of 
the  dlatance  d  and  ao  we  may  apaak  of  the  function  P^(d).  The  random 
element  ia  inpoaad  in  the  problem  by  coneidering  a  random  variable  (d) 
(alao  a  function  of  dlatance)  which  la  defined  aa  the  relative  frequency 
of  detection  of  camouflage  1  at  dlatance  d.  The  reaponae  model  la  then 
x^ (d)  -  P^d)  +  e  where  the  non-random  element  P^(d)  la  a  maaaure  of  loca¬ 
tion  of  the  random  variable  x^d).  We  can  anvialon  the  2  probability 
curvea  aa  follows: 


Theaa  curves  have  the  properties  of  P^O)  -  1,  P^t")  ■  0,  and  P£(d)^  0 
(monotonic  decreasing),  Perhaps  normal,  logistic,  or  exponential  models: 
Pl(d)  -  <-^>0cd)2 

Pi(d)  -  4ekd/  <l+ekd)2  ,  k>o 


P^d)  -  e 


-kd 


can  be  fitted  in  order  to  approximate  the  non-random  curves  P^d), 
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1  believe  chat  two  aaparata  problems  ara  now  distinguishable! 

1)  Which  camouflage  la  batter?  (the  anawar  to  thia  of  courae  depanda 
on  our  critarion  of  goodnaaa  of  camouflage))  and  2)  What  la  F^(d)  (tha 
probability  of  datactlon  ualng  camouflage  1)  at  a  given  dletance  d?  The 
experiment  aa  praaented  la,  I  believe,  much  batter  aultad  to  anawar 
question  2  and  rather  poorly  aultad  to  anawar  question  1. 

The  following  apedfic  auggaationa  may  ba  oj^ame  use  In  regard 
to  tha  experiment  aa  performed  or  to  future  experlmenta.  Change  the 
raaponae  from  many  nulla  and  asms  distances  (corresponding  to  tha  ques¬ 
tion  "At  what  distance  do  you  detect  the  camouflage?")  to  all  O'o  and  l'a 
(for  NO  and  YES  respectively)  corresponding  to  tha  question  "Do  you  detect 
tha  camouflage?" 

The  present  response  aa  given  in  the  Teat  Results  table  reflects 
too  much  on  tha  quaatlom  "What  does  P^d)  look  like  at  this  d?"  and  not 
enough  on  the  null  hypothesis  question!  "Does  P^(d)  -  P2 (d)  for  this 
(relevant)  d?"  The  data  In  the  ant  Result*  table  are  effectively  for 
tha  two  dlatancaa  d  «  1772  and  d  -  1032,  1  believe  thsx  it  la  proper  in 
thlm  aituation  to  split  up  the  data  (one  subset  for  each  altitude  or  dis¬ 
tance)  and  make  two  aaparate  testa  of  hypothesis!  Hft!  P^d^  ■  P^d,)  and  Hq! 
Pl(d2)  ■  P2(d2>.  If  the  teat*  prove  to  be  significant  in  the  same  direc¬ 
tion  at  both  distances,  then  this  would  strongly  indicate  that  one  camou¬ 
flage  Is  more  affective  than  the  other. 

For  a  fixed  distance,  we  are  essentially  comparing  two  probabilities 
In  which  case  the  Sign  Teat  (see,  for  example,  Dixon  and  Massey,  2nd 
edition,  page  280)  is  an  excellent  way  of  proceeding.  Complications  do, 
however,  arise  from  the  presence  o£  zeros  (ties)  in  the  Sign  Test  differences. 

Applying  the  Sign  Test  procedure  to  the  high  altitude  (d  *  1772) 
data  we  get  2  plusses,  13  zeros,  and  1  minus.  It  ms Draa  a  difference  (even 
intuitively)  whether  there  are  2  plusaep,  0  zeros,  and  1  minus  as  opposed 
to  2  plusses,  10®  zeros,  and  1  mlrus.  The  relative  number  of  zeros  gives 
an  indication  of  the  height  of  the  P^(d)  curves  and  of  tha  relative 
difference  of  the  P^(d)  curves  at  this  d.  Many  zeros  formed  from  1-1 
differences  indicate  that  the  F^(d)  curves  are  near  1  and  close  to  one 
another.  Many  zeros  formed  from  0-0  differences  indicate  that  the  P^(d) 
curves  are  near  o  and  dose  co  one  another.  In  the  present  case,  the 


-399- 


relatively  large  number  of  zeroa  (13),  all  of  which  were  formed  from 
0-0  differencea,  indicates  that  the  P^ (d)  curves  are  near  zero  and  re¬ 
latively  close  for  this  distance  d. 

The  presence  of  zeroa  (ties),  chough  somewhat  informative,  does 
invalidate  the  calculation  of  the  standard  binomial  probabilities.  To 
get  around  this  complication,  I  propose  the  following  Modified  Sign  Test 
which  take  into  account  information  on  ties.  As  before,  we  are 
interested  in  the  null  hypothesis  Hqi  P^(d)  ■  Pj(d),  where  d  is  fixed. 
After  taking  differences,  as  we  would  in  the  standard  Sign  Test,  the 
above  null  hypothesis  is  converted  intot  Hqj  P_^_  -  P_(i.e„  the  probability 
of  a  positive  difference  equals  the  probability  of  a  negative  difference). 
However,  if  ties  exist,  then  the  probability  of  a  zero  difference  (PQ) 
also  exists— -let  us  call  this  p  (i.e.,  PQ  -  p),  p  is  a  nuisance  parameter 
that  is  a  function  of  the  height  and  the  relative  difference  of  the  P^(d) 
curves  for  this  fixed  d.  Although  p  is  unknown,  let  us  momentarily  treat 
it  as  known  and  fixed.  The  probability  of  exactly  i  +  1  s,  J  o'c,  and  k-'s 
is  therefore,  under  the  null  A^othesls ,  given  by  the  trinomial  expression 


PHi.j.k)}  -  ffijjfoq  (P+)i(P0)J(P.)k 


which  may  be  written ;in  terms  of  p)  as 


P{(i,j,k)  }■ 


llllk! 

<!i+Jik): 


(since  Pq  ■  p  by  definition,  P+  »  P_  under  Ho,  and  P+  +  Po  +  P_  ■  1), 

Using  the  above  probability  expression,  we  can  then  compute  the 
tail  probability  by  simple  summation  over  th region;  this  tall 
probability  will  also  be  a  function  of  p.  It  is  now  suggested  that  we 
maximize  (either  analytically  or  numerically)  this  tail  probability  with 
respect  to  p.  If  this  maximum  is  still  lesa  than  .05,  for  Instance,  then 
a  valid  statistically  significant  result  has  been  obtained  at  the  5Z 
level.  Some  minor  difficulties  arise  in  defining  an  appropriate  tall 
region  due  to  the  loss  of  a  natural  ordering  in  2  or  more  dimensions; 
however,  careful  consideration  of  various  possible  outcomes  quickly 
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lead*  us  to  a  realisation  of  what  constitutes  a  "worse  event"  than  what 
was  observed.  Application  of  the  above  Modified  Sign  Test  to  the  authors' 
data  yields  a  non-significant  result  at  the  5X  level.  As  indicated  before, 
this  is  not  surprising  because  even  though  we  now  have  the  analytical 
tools  to  carry  out  the  test,  it  is  atill  felt,  for  this  small  sample  size, 
that  the  magnitude  and  differences  of  the  P^(d)  curves  are  too  small  to 
make  a  significant  result  likely  to  occur. 

A  final  suggestion  is  to  change  the  experiment  outright  so  as  to 
perhaps  answer  more  directly  the  question  "Which  camouflege  is  better?" 
This  could  be  done  by  1)  physically  setting  up  two  camouflages  side-by- 
side  at  each  site  (rather  than  having  one  camouflage  at  one  site  and  the 
other  camouflage  at  the  other  site)}  2)  telling  the  observer  exactly  where 
the  sites  are  so  that  he  will  definitely  sea  the  two  side-by-side  camou¬ 
flages}  and  3)  changing  the  question  to  be  asked  from  "At  what  distance 
to  you  detect  the  camouflage?"  to  "Which  of  the  two  side-by-side  camou¬ 
flages  do  you  consider  more  easily  detectable?" 

Proceeding  in  this  manner,  one  can  ensure  the  absence  of  null 
responses  in  which  case  the  Sign  Test  is  then  directly  applicable.  We 
note  in  passing  that  for  sample  size  n  *  16,  as  is  the  Meyer-Johnson 
sample  size,  at  least  12  of  the  responses  would  have  to  be  of  the  same 
type  for  significance  at  the  51!  level. 

On  the  other  hand,  a  bit  more  generality  can  be  added  to  the 
testing  procedure  by  not  constraining  the  observer  to  say  each  and  every 
time  that  camouflage  1  is  definitely  more  conspicuous  than  camouflage  2 
(or  vice  versa).  It  is  conceivable  and  realistic  that  an  observer  may 
respond  to  the  question  "Which  of  the  two  side-by-side  camouflages  do 
you  consider  more  easily  detectable?"  by  "Neither  looks  better  than  the 
other—they  both  appear  to  be  Just  about  equally  (in) conspicuous,"  If 
this  type  of  response  is  permitted  then  the  Modified  Sign  Test  as  des¬ 
cribed  previously  would  be  applicable. 
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DESIGN  OF  RELIABILITY  EXPERIMENTS  TO 
YIELD  MORE  INFORMATION  ON  FAILURE  CAUSES 


Roland  H.  Rigdon 

U.  S,  Army  Weapon#  Command,  Rock  Island,  Illinois 


The  dealgn  of  reliability  experiments  should  be  changed  to 
separate  the  main  modes  of  failures.  This  can  be  done  by  comparing 
failures  and  variable  Input  factors  such  as  hours  of  operations,  miles, 
rounds  fired,  or  circuit  actuations. 

Each  test  mission  or  test  day  should  be  considered  as  a  separata 
treatment  of  the  factors  at  a  high  or  low  level.  The  structure  Is  a 
two-level  factorial.  The  conditional  probabilities  of  different  pro¬ 
portions  of  failures  observed  at  high  levels  of  the  factors,  given 
that  a  relationship  exists  betwean  failures  and  the  factor,  can  be 
pre-determined.  This  conditional  probability  is  inverted  to  yield  a 
conditional  probability  that  a  relationship  exists  given  the  failure 
combinations  observed  during  the  test. 

The  failures  may  be  related  to  uptime  in  which  case  they  are 
usually  mutually  independent  failures  and  are  used  to  predict  Mean  \ 

Time  Between  Failures  (MTBF)  or  the  failures  can  be  used  to  predict 
Mean  Miles  Between  Failures  (MMBF)  or  Mean  Rounds  Between  Failures 
(MRBF) . 

The  dealgn  engineer  will  segregate  failures  into  mobility  and 
non-mobility  failures.  Mobility  failures  of  the  suspension  and  drive 
train  are  dependent  upon  the  stresses  resulting  from  the  application 
of  drive  forces  and  terrain  forces  and  may  also  be  wear-out  related. 

If  a  nan-mobility  part  has  failures  related  to  mileage,  the 
design  engineer  knows  that  the  failures  may  be  due  to  vibration, 

This  is  especially  true  of  a  tracked  vehicle  because  the  chordal 
action  of  movement  Induces  a  longitudinal  vibration  of  a  frequency 
linearly  related  to  ground  speed,  Thua,  acceleration  from  minimum 
to  maximum  speed  will  induce  an  increasing  frequency  chat  will 
coincide  at  some  point  with  the  resonant  frequencies  of  most  parts. 

If  the  failures  are  predicted  as  Mean  Rounda  Between  Failures 
(MRBF)  the  design  engineer  knows  that  the  failures  may  be  caused  by 
the  high  translation  acceleration  of  the  recoil  impulse.  Also, 
because  this  firing  pulse  Is  an  impulse  the  falling  part  may  be 
excited  to  vibrate  at  its  resonant  frequency  and  induce  failure. 

Most  electrical  and  electronic  failures  arc  predicted  as  a 
function  of  uptime  (MTBF).  However,  most  failures  of  such  equipment 
occur  at,  or  are  induced  by,  the  mechanical  interfaces.  Since  the 
mechanical  components  are  in  steady  state  during  uptime  and  in  dynamic 
state  at  energization  and  deenergization,  many  failures  may  be  related 
to  the  number  of  circuit  activations , 
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The  above  diacuaelon  reveala  the  dealgn  engineer' a  lntereat  In 
relating  failures  to  hours  uptime,  milaa,  rounds  fired,  and  circuit 
activations.  He  muat  have  thia  information  if  he  la  expected  to 
redesig  >  a  part  to  reduce  the  failure  rate  and  achieve  that  redesign 
at  a  low  risk. 

Before  discussing  the  proposad  design  of  experiments,  we  can 
discuss  some  prior  assumptions.  It  is  assumed  that  the  Reliability 
Block  Diagram  and  the  Failure  Modes  and  Effects  Analysis  (FMEA)  have 
been  completed  prior  to  designing  the  reliability  tests.  The  failures 
are  assumed  to  be  independant.  Since  the  lambda  of  a  Poisson  distribution 
would  be  very  small,  the  probability  of  two  failures  on  one  part  during 
one  mission  approaches  zero.  Should  two  or  more  failures  occur  during 
one  mission,  it  is  possible  that  the  failures  are  not  Independent,  i.e., 
a  maintenance  failure  has  occurred  in  the  restoration  of  the  previous 
failure.  Or  the  lambda  parameter  of  the  Poisson  distribution  may  not 
be  small. 

Most  parts  will  have  only  two  or  three  modes  of  failure.  In  real 
life  situations,  if  many  failures  are  experienced  on  a  given  part, 
most,  if  not  all,  will  be  of  the  aame  failure  mode  because  of  a  week 
link  in  the  design.  This  consistency  of  failure  will  be  obvious 
evidence  for  the  design  engineer  to  use  in  his  redesign.  The  Dealgn 
of  Experiments  herein  proposed,  will  be  moat  valuable  In  classifying  a 
small  number  of  failures  with  respect  to  levels  of  the  varied  test  factors. 

The  design  of  present  reliability  tests  is  based  upon  the  weapon 
system's  mission  profile  with  a  48  hour  battlefield  day  ea  a  typical 
mission.  During  this  48  hour  day,  the  system  will  be  operating  between 
18  and  24  hours  and  travel  a  specific  number  of  mlleo  while  firing  a 
specific  number  of  rounds.  Present  practice  Is  to  establish  a  test 
mission  and  then  perform  this  one  mission  over  and  over  again  to 
accumulate  the  desired  number  of  hours,  miles  and  rounds  on  the  weapon 
system. 

Each  weapon  unit  should  be  tested  over  a  sufficient  number  of 
missions  so  that  the  accumulated  hours  or  miles  exceeds  the  weapon 
system's  expected  life.  If  this  is  not  possible,  the  accumulated 
interval  should  at  least  equal  the  expected  intervals  between  rebuilds  or 
overhauls.  Only  with  this  long  an  interval  of  test,  can  wear-out  data  be 
obtained. 

The  distribution  function  of  failures  can  be  expressed  in  the 
dimensions  of  hours,  miles,  or  rounds,  If  all  the  test  missions  are 
identical,  the,  ratios  among  these  dimensions  will  remain  constant.  Even 
with  some  pri<3r  Information  that  a  particular  failure  Is  related  to 
miles,  .the  achieved  failure  distribution  expressed  in  miles  cannot  be 
compared  to  the  failure  distribution  expressed  in  hours,  Therefore, 
there  is  no  test  of  the  hypothesis  that  a  failure  to  miles  relation¬ 
ship  actually  exists. 
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To  overcome  this  d Henna  and  to  provide  data  to  the  statistician, 
a  new  deaign  of  the  reliability  teata  ia  propoaed.  The  atructure  of  the 
test  deaign  should  be  such  chat  not  all  miaeions  are  identical.  Time 
would  remain  constant  ovmr  all  missions,  but  other  factors  of  interest 
would  be  varied  to  conioro  to  a  two-level  factorial.  As  an  example, 
a  present  design  may  test  all  missions  running  100  miles  in  24  hours. 

The  proposed  design  would  run  half  o^-^e  missions  over  50  miles  ih 
24  hours  and  half  over  150  miles  in  24  hours.  The  average  is  still 
100  miles  per  mission.  If  the  failures  are  related  to  time,  the 
probability  of  experiencing  a  failure  in  150  miles  is  equal  to  the 
probability  of  experiencing  a  failure  in  50  milea,  but  if  the  f allurea 
are  related  to  miles,  the  probability  of  failure  in  150  miles  is  three 
times  that  in  50  miles. 

The  basic  factorial  design  must  be  altered  to  prevent  wear-out 
failures  being  confounded  with  failures  related  to  a  factor  of  interest. 
The  design  of  experiment  should  be  structured  in  such  a  way  that  at 
any  point  in  a  unit's  life  the  accumulated  number  of  missions  at  high 
and  low  levels  should  be  about  equal . 

Appendix  1  contains  a  design  for  a  hypothetical  tank  test.  An 
initial  factorial  design  is  shown  in  Table  I  and  then  altered  in  Table 
II  to  remove  the  confounding  of  wear  -out  failures. 

The  analysis  requires  splitting  the  treatments  into  that  half  with 
a  high  level  and  that  half  with  a  low  level  of  the  factor.  Than  the 
number  of  failures  occurring  during  high  level  missions  can  be  compared 
to  the  total  number  of  failures  and  indicated  as  F^,  where  1  varies  from 

0  to  the  total  number  of  failures.  If  is  the  hypothesis  that  the 
failures  are  independent  of  the  factor  and  is  the  hypothesis  that 

the  failures  are  dependent  on  the  varied  factor  of  interest,  a  set  of 
conditional  probabilities  of  FA  given  Hj  can  be  constructed  as  shown 

in  Table  IV  of  Appendix  II. 

However,  the  data  from  a  test  are  the  observed  values  of 
while  the  information  of  interest  is  the  conditional  probability  that 
the  is  true  given  the  observed  failure  combination  F^.  The 
conditional  probabilities  must  be  inverted. 

The  inversion  of  the  conditional  probability  can  be  accomplished 
using  Bayes'  Theorem  as  follows) 

Pr  (Hj|F1)  -  rr  (FjH£  Pr  (H^) 

S  Pr  (K.|H  )  Pr  (H  ) 

J  3  3 
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Of  interest  In  this  case  is  the  conditional  probability  that  the 
hypothesis  that  failures  are  dependent  on  the  factor,  or  Hj,  is  true 
given  the  observed  combination  of  failures.  * 

Prior  to  analysis  using  Bayes*  Theorem,  an  a  priori  assumption  of 
H2  must  be  made.  This  can  be  established  at  a  neutral  point  of  ,5 

or  can  be  varied  dependent  upon  information  from  previous  tests,  or  the 
a  priori  assumption  can  even  be  established  from  previous  engineering 
analysis  of  the  failures  considered  in  the  combination.  This  allows 

all  available  information  to  enter  into  the  analysis. 

Appendix  II  contains  several  tables  of  the  conditional  probability  of 
(H2|Ft).  The  desired  table  is  selected,  depending  upon  the  level  of  the 
assumed  a  priori  probability  of  Hj,  Using  the  observed  combination  of 

failures  (F^)  the  conditional  probability  that  is  true  given  the 

failure  combination  F^  can  be  read  directly  from  the  tables.  The 

statistician  can  then  Inform  the  design  engineer  of  a  definite  probability 
that  the  failures  are  related  or  unrelated  to  specified  factors.  Appendix 
III  contains  examples  of  the  use  of  different  e  priori  assumptions. 

Note  that  time  as  a  factor  has  been  held  conatant  over  all  missions.  If 
no  relationship  can  be  determined  for  failures  during  reliability  analyses, 
the  failures  are  assumed  to  be  random  and  are  customarily  measured  over  a 
time  base.  Also,  reliability  must  be  measured  on  a  time  base  to  compute 
availability. 

Another  reason  for  using  a  constant  time  is  to  simplify  testing.  The 
48  hour  battlefield  day  of  the  mission  profile  can  usually  be  compressed 
into  a  32  hour  teat  day  in  the  test  profile.  This  suggests  the  use  of  two 
16  hour  days  of  two  8  hour  shifts  etch.  It  also  affords  the  opportunity 
to  conduct  both  day  and  night  testing. 

Examination  of  Appendix  I,  a  sample  test  design,  discloses  a 
disadvantage  of  this  suggeBtsd  design.  Under  present  designs  only  one 
test  profile  or  mission  must  be  structured  for  several  repetitions. 

The  new  design  may  require  as  many  as  32  tsst  profiles.  It  is  recommended 
that  test  modules  of  5  miles  each  be  designed.  Different  modules  would  have 
different  speeds,  terrains,  number  of  rounds  fired,  etc.  A  test  profile 
of  30  miles  would  then  be  the  sum  of  10  or  more  modules  while  a  150  mile 
profile  could  contain  30  or  more  modules,  some  repeated. 

Whatever  methods  are  used  to  structure  the  reliability  test,  the  design 
ehould  provide  data  In  such  a  form  that  It  can  indicate  both  what  part  failed 
and  aome  conditional  relationships  with  the  test  environment. 


-406- 


APPENDIX  I 


HYPOTHETICAL  TANK  RELIABILITY  TEST 


Assume  a  mission  profile  for  the  48-hour  battlefield  day  as  follows) 

24  hours  run  time 
100  miles  movement  (M) 

20  rounds  of  main  armament  (full  charga)  (R) 

120  electrical  circuit  actuations  (A) 

200  rounds  of  cupola  machine  gun  (C) 

200  rounds  of  co-ax  machine  gun  (X) 

The  electrical  circuit  actuations  are  the  energisation  of  all  intermittent 
circuits  in  an  established  sequence. 

Expected  life  of  the  weapon  system  at  overhaul  is  5,000  miles  no 
conduct  approximately  50  missions  on  each  tank  to  acquire  wear-out  data. 
Replicate  with  a  sample  of  five  tanka  for  a  total  of  25,000  miles. 

Construct  an  initial  factorial  design  as  shown  in  Table  I  using  the 
following  levels  of  each  factor t 

Constant  •  24  houra  run  time 
M  -  50  and  150  miles 
R  -  10  and  30  rounds 
A  -  60  and  ISO  actuations 
C  »  100  and  300  rounds 
X  -  100  and  300  rounds 

Reorder  the  runs  of  the  initial  factorial  into  the  altered  factorial  of 
Table  II.  This  will  remove  the  effects  of  wear  from  the  analysis. 

The  total  of  48  missions  per  tank  will  yield  24,000  miles, 

5,760  hours,  4,800  main  gun  rounds,  28,600  actuations,  48,000  cupola  gun 
rounds  and  48,000  coaxial  gun  rounds.  The  expected  total  number  of 
failures  would  range  from  190  to  240  on  the  system.  Table  III  shows  the 
different  levels  of  a  reliability  block  diagram  with  the  range  of  the 
number  of  blocks  at  each  level  and  some  ranges  of  expected  number  of 
failures  of  a  particular  block  to  warrant  investigation. 


TABLE  §  I  5  5 

INITIAL  FACTORIAL  DESIGN  AT  (2  )+(i)(2  ) 
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APPENDIX  II 


Bayesian  Probabilities  of  Dependency 


The  fsilures  of  the  test  are  analysed  to  determine  the  probabilities 
that  one  of  the  following  hypotheses  Is  true  (usually  R2)t 

-  The  failures  are  not  dependent  upon  the  factor  varied 

-  The  failures  are  dependent  upon  the  factor  varied 
and  H2  are  mutually  exclusive,  collectively  exhaustive 

Table  IV  lists  the  conditional  probabilities  of  F^  given  the 

hypothesis  la  true.  The  symbol  represents  the  number  of 

failures  during  those  half  of  the  missions  treated  at  the  factor's 
high  level  In  combination  with  the  total  number  of  failures.  Note  that 
the  combination  is  represented  as  a  ratio  but  »  5/10  +  3/6  jt  1/2 

because  the  total  number  of  failures  must  always  be  considered. 

Most  parts  will  have  only  two  or  three  or  at  the  most  five  or  six 
modes  of  failure.  If  there  are  more  than  ten  failures,  a  consistency  in 
the  mode  of  failure  can  be  seen  which  can  be  analysed  to  determine  the 
redesign.  Therefore,  the  tables  are  only  completed  through  ten  failures. 

Table  IV  the  conditional  probability  of  F^,  was  constructed  under 

the  assumption  that  Hj  was  true.  But  after  the  test  is  conducted  the  data 

contains  only  P.  so  that  H.  must  be  determined  from  this  data.  Bayes' 
Theorem  can  be  stated  as:  3 

Pr  (fJh.)  Pr  (H.) 

)  -  - i- J - i~ 

Z  Pr  (F.  |  H, )  Pr  (H.) 

3  3  3 

J  ■  1,2 

1  *  1, 2, . , . ,n,  n+1 
n  -  total  number  of  failures 
Pr  (Hx)  +  Pr  <H2)  -  1.0 


Pr  (H.|F1 
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BAYESIAH  Pr  (HglPj),  Eq  -  -7  a  priori 


cN 


to 


VO 


w 


•  • 


VO 


•  • 


s 

R 

R 

o 

o 

• 

H 

■ 

On 

o 

S' 

• 

d 

• 

§ 

■t 

• 

• 

3 

• 

5j 

o 

m 

r 

H 

• 

<n 

• 

IA 

• 

t- 

f- 

w 

• 

R 

• 

£ 

• 

SB 

LA 

R 

LA 

00 

• 

« 

t*T 

0J 

R 

£ 

• 

Si 

• 

l-~ 

Pi 

§ 

r— 

Oi 


to  VO  o 

V)  M  4 

ro  VO  oo 

•  •  • 


i 
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APPENDIX  III  -  Examples 


Example  I. 

Assume  a  tank  weapon  system  Is  being  tested  as  described  In  Appendix 
I.  During  the  24,000  miles,  5,750  hours,  and  4,600  gun  rounds  a  total 
of  five  failures  was  experienced  on  road  wheel  arms. 

Because  this  Is  the  first  test  undsr  this  method  of  testing,  no 
prior  assumptions  are  made  about  relationships  between  failures  and  miles, 
rounds,  or  hours.  Therefore,  the  a  priori  probabilities  are  considered 
neutral  or  .5. 

One  of  the  failures  of  the  road  wheel  arm  occurred  during  a 
firing  exercise.  Another  failure  was  detected  immediately  after 
a  firing  exercise.  No  information  is  known  about  the  other  three 
failures.  All  existing  evidence  therefore  indicates  a  relationship 
between  failures  and  rounds  fired. 

The  a  priori  probability  of  .5  can  now  be  altered.  Assume  a 
probability  weighting  of  1.0  for  the  two  failures  related  to  rounds 
fired  and  a  weighting  of  ,5  for  the  other  three.  Thus,  an  a  priori 
probability  of  .7  (3. 5/5.0)  la  established. 

The  total  failures  occurring  during  missions  of  30  rounds  firing 
(high  level  of  rounds)  is  found  to  be  only  two  of  the  total  of  five, 
Inspection  over  the  Table  VIII  matrix  of  the  column  under  five  total 
failures,  two  of  wh\.ch  are  at  the  high  level,  indicates  a  conditional 
probability  of  ,396  that  the  failures  are  related  to  rounds  fired,  given 
that  two  of  the  five  occur  at  the  high  level  of  rounds  during  a  mission, 
and  given  an  a  priori  probability  of  ,7, 

The  relationship  of  failures  to  miles  is  now  examined.  A  total  of 
four  of  the  five  failures  are  found  at  the  high  level  of  miles  per 
mission.  Using  an  a  priori  probability  of  .5,  Table  V  yields  a 
conditional  probability  of  .716  that  the  failures  are  related  to  miles 
(conditional  probability  of  ,284  that  the  failures  are  related  to  time). 

This  statistical  evidence  thus  indicates  that  the  road  wheel  arm 
failures  should  be  measured  as  Mean  Miles  Between  Failures  (MMBF)  despite 
the  fact  that  the  only  available  engineering  evidence  indlcatee  measurement 
as  Mean  Rounds  Between  Failures  (MRBF) . 

Failure  due  to  rounds,  or  impulse  loading,  may  require  an  entirely 
different  fix  than  would  a  failure  due  to  miles,  or  cyclic  loading. 
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Example  II, 


Aaeuma  the  tank  weapon  system  of  Example  I  wae  the  first  to  be 
tested  under  this  new  procedure.  The  second  weapon  system  to  be  tasted 
is  an  armored  personnel,  carrier.  The  road  wheel  arms  are  of  a  similar 
design  and  construct  and  have  a  similar  strength  to  stress  ratio. 
Xherafore,  the  results  of  the  tank  test  are  used  to  establish  an  a 
priori  probability  of  .7  that  failures  are  related  to  miles  and  an  a 
priori  probability  of  .4  that  failures  are  related  to  rounds. 

A  total  of  four  road  wheel  arm  failures  are  experienced  with 
three  at  the  high  level  of  miles.  Table  VIII  indicates  a 
conditional  probability  of  failure  relationship  to  miles  of  .797,  Two 
of  the  four  failures  occurred  at  the  high  level  of  rounds  and  Table  X 
Indicates  a  conditional  probability  of  failure  relationship  to  rounds 
of  .272. 

The  third  weapon  system  to  be  tested  is  a  reconnaissance  vehicle 
about  the  weight  of  the  armored  personnel  carrier  but  with  a  main  gun 
firing  impulse  nearer  that  of  the  tank.  The  road  wheel  arms  are 
designed  stronger  to  take  this  impulse. 

The  results  of  the  second  test  are  used  to  establish  a  priori 
probabilities  of  and  .3.  A  total  of  eight  failures  are 

recorded  of  Milch  five  are  at  the  high  level  of  miles  end  six  are  at 
tin.  high  level  of  rounds.  These  yield  conditional  probebilitlas  of 
.792  for  e  relationship  to  miles  (Table  VII)  and  ,549  for  a  relation¬ 
ship  with  rounds  (Table  XI). 

This  is  counter  to  all  previous  evidence.  Disregarding  the 
previous  evidence  end  re-establishing  a  neutral  a  priori  probability  of 
.5,  Table  V  yields  conditional  probabilities  of  ,487  and  .740  for  the 
failure  relationship  with  miles  and  rounds  respectively.  This  leads 
to  accepting  the  failures  as  related  to  rounds  with  a  conditional 
probability  of  .740,  The  conditional  probability  of  relationship  to 
miles  is  nearly  neutral,  so  previous  data  can  be  accepted  and  the 
conditional  probability  of  failure  relationship  to  miles  is  stated  as 
.792,  Thus,  any  redesign  to  reduce  failures  should  examine  both  MMBF 
and  MRBF  for  causes. 


Example  111. 


During  testing  of  the  tank  in  Example  I,  a  "black  box"  in  the 
turret  has  experienced  ten  failures.  The  box  contains  five  electricsl 
circuits  and  due  to  lack  of  circuit  protection  each  failure  of  the  box 
has  resulted  in  the  failure  of  from  tvo  to  all  five  circuits.  As  a 
consequence,  the  initial  failure  mode  cannot  be  Isolated  from  its  effacta 
on  other  circuits. 


An  accelerometer  ha*  bean  used  to  measure  the  vibration  at  the 
surface  of  the  "black  box".  Resonant  frequency  was  30  H  at  a  ground 

Z 

speed  of  12  MPH  and  with  ,5g  lateral,  l.Og  longitudinal  and  3,0g  vertical 
amplitudes.  The  transient  loading  from  gun  fire  shock  was  2,0g  over 
10  mllllaeconds. 

Prom  the  above  data,  a  tentative  decision  Is  made  to  isolate  the 
box  In  the  vertical  direction,  This  decision  Is  based  upon  the  only 
data  available  under  standard  test  procsdures. 

However,  the  new  procedure  allows  an  evaluation  of  the  relationship 
between  failures  and  miles  (vibration)  and  rounds  (gun  shock).  A 
neutral  assumption  of  a  priori  probabilities  of  .5  is  chosen.  Five  of 
the  ten  failures  were  recorded  at  the  high  level  of  miles  and  eight  were 
recorded  at  the  high  level  of  rounds  per  mission. 

Using  these  values  to  enter  Table  V,  the  matrix  yielde  conditional 
probabilities  of  .192  and  ,865  for  the  relationships  of  failures  to 
miles  and  rounds  respectively.  This  is  a  strong  indication  that  the 
gun  shock  transient  is  forcing  the  components  inside  the  box  into  frae 
resonant  vibration.  Examine  each  component  to  determine  which  ones  can 
vibrate  or  displace  under  the  unidirectional  impulse  of  gun  shock.  A 
solution  may  be  to  rotate  some  parts  90  degrees  to  move  the  forcing 
function  out  of  their  vibrational  axes. 

Note  that  in  this  example,  the  tentative  solution  of  isolation 
in  the  vertical  direction  may  not  reduce  failures  and  may  actually 
Increase  failures. 
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MAXIMUM  LIKELIHOOD  APPROXIMATION  FOR  GUMBEL'S  LAW  AND 
APPLICATION  TO  UPPER  AIR  EXTREME  VALUES 

Oskar  Essenvangar 

Phyaical  Sciancaa  Directorate 
Diractorata  foe  Research,  Development,  Engineering 
and  Mlaaile  Systems  Laboratory 
U.  S,  Army  Mirella  Command 
Radatona  Arsenal,  Alabama 


ABSTRACT 4  Two  typea  of  extrema  value  diatributiona  are  commonly 
applied  and  have  bean  employed  to  fit  extreme  valuer  of  upper  air  data  of 
wind  spaed,  temperature,  and  denaity.  Gumbel'a  (or  Fieher-Tippett  I) 
dlatrlbutlon  fits  well  in  general,  while  the  Flaher-Tippett  II  dietrlbutlon 
display*  aignlf leant  davlationa  of  the  obaervad  data  irom  the  analytical 
curve  for  temperature  and  denaity  whan  judged  by  the  Kcfaogorov-Smlrnov 
teat. 


Gumbel'a  law  requlrea  two  paramatera  to  ba  aatimated  from  the 
observed  data  x^  Momenta  estimators  for  ct  and  u  are  readily  available, 

but  not  very  efficient.  The  maximum  likelihood  estimators  a  end  u  can 
be  computed  by  Iteration  methods,  but  usually  not  without  electronic 
data  processing,  Lieblein  haa  introduced  minimum  variance  linear  order 
atatistlcs  estimators,  which  need  table  values  (weighting  factors  of 
the  observations)  and  necessitate  keeping  the  data  in  sequence  of 
recording  (i.e.  time). 

The  author  has  developed  and  tested  two  methods  from  which  maximum 
likelihood  estimators  can  be  obtained  by  modification  of  the  likelihood 
equations,  but  based  on  mere  ranking  of  data.  The  a  is  computed  from  an 
analytical  expression  rather  than  the  lengthy  iteration  process  ordinarily 
necessary  to  solve  the  likelihood  equations.  The  approximation  of  a  and  u 
by  the  two  methods  takes  a  minute  fraction  of  the  computer  time  compared 
with  the  time  for  the  iteration  of  the  maximum  likelihood  equations. 
Although  the  second  method  leads  to  a  quadratic  equation,  one  solution 
can  readily  be  discarded  by  consideration  of  u. 

The  methods  were  tested  against  the  results  of  the  maximum  likelihood 
solution.  It  was  found  that  for  all  three  atmospheric  parameters  (wind 
speed,  temperature,  density)  at  altitude  levels  1  km  through  24  km  (in  1  km 
steps)  for  distributions  of  summer  and  winter  data  the  approximation 
methods  provided  excellent  agreement  with  the  true  a  and  u  estimators. 

1.  INTRODUCTION.  A  variety  of  factors  must  be  considered  in  the 
assessment  of  the  atmospheric  influence  upon  missiles  and  rockets.  It 
is  self-evident  that  extreme  values  of  atmospheric  parameters  weigh  heavily 
in  the  analysis  of  the  atmospheric  effect.  Of  great  Importance  to  the 
engineer  is  therefore  the  proper  representation  of  the  distribution  of 
extreme  values,  especially  of  upper  air  data. 
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Two  types  of  axtrama  value  distribution  would  mainly  be  applicable 
to  fit  upper  air  data  of  wind  speed,  temperature  and  density,  namely 
Fisher-Tippet t  I  arid  II  (1928).  Although  other  frequency  dietributions 
could  be  employed,  these  two  typas  appear  best  suited  to  represent  the 
above  mentioned  data.  The  Flsher-Tippett  I  distribution  has  also  been 
derived  by  Gumbel  (1958)  as  tha  limiting  distribution  of  the  m~th  dis¬ 
tribution,  and  is  often  referred  to  as  Gumbel ' s  distribution.  An  later  . 

demonstrated,  Cumbel's  curve  fits  well  in  general,  while  tha  Flsher-Tippett  II 
distribution  displays  signlficsnt  deviation  of  the  observated  data  from 
the  analytical  curve  at  various  altitude  levels.  Tha  significance  was 
tested  by  the  Kolmogorov-Smlrnov  test  (see  Kolmogorov  1933,  and  Smirnov, 

1948). 


From  a  theoretical  point  of  view  in  statistical  analysis,  maximum 
likelihood  estimators  would  ba  best  for  determination  of  tha  parameters 
from  the  observed  data.  These  estimators  cannot  be  expressed  in  explicit 
form,  however.  Iteration  methods  are  usually  applied  to  solve  for  the 
ertimatora.  These  computations  can  be  expensive  when  a  large  number  of 
samples  is  involved.  Although  Liebleln  (1954)  has  introduced  minimum 
variance  linear  order  statistics  estimators,  his  method  requires  that 
the  observations  are  kept  in  sequence  (e.g.  time).  A  further  dis¬ 
advantage  for  computer  use  of  the  latter  is  the  determination  of  the  length  of 
subsections.  The  author  has  therefore  attempted  to  derive  estimators 
which  are  based  on  the  maximum  likelihood  equations  and  are  close 
approximations  to  the  maximum  likelihood  estimators.  In  essence  for 
N  ■>  «  they  would  be  identical  with  them.  Since  the  computation  of 
these  estimators  does  not  require  iteration,  sequence  of  data  or  deter¬ 
mination  of  adequate  subsections,  the  deficiencies  of  the  existing 
methods  in  computer  applications  are  resolved.  It  is  proven  with  the 
data  samples  on  hand  for  upper  air  data  that  the  estimators  are  in 
excellent  agreement  with  the  true  maximum  likelihood  estimators,  and 
the  computation  is  less  costly  than  with  any  other  method  except  the 
moments  estimators.  The  derivation  and  detailed  results  are  presented 
In  the  following  sections. 


2.  ESTIMATION  OF  GUMBEL* S  DISTRIBUTION.  The  Gumbel  extreme  value 
distribution  can  generally  ba  written  in  the  form  of  the  cumulative  dis¬ 
tribution 

F(s)  -  exp  t  -  exp  (-*)]  (1) 

with  the  normalized  variable  z  related  to  the  observed  variate  x  by 

z  “  a  (x  -  u).  (2) 

The  two  constants  a  and  u  are  scale  and  reference  parameters,  respectively, 
and  must  be  determined  from  the  observations  by  estimation  procedures. 
Several  methods  to  derive  estimators  are  available. 
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a.  Moments  Estimators 


It  is  assy  to  prova  that 

*  “  V*x  (3 
with  0^  »  1.28254  and  sx  «x,  tha  usual  moments  estimator  for  tha 
standard  daviation.  Hanca  tha  moments  astlaator  Is 


o^,  -  1.28234/Sj 


Further 


u  ■  x  +  s/a  (4) 

u  -  x  +  s  •  ax/aB  (4a) 

u  ■  x  +  0.45005  ox«  (4b) 

As  usual,  tha  naan  value  Is  replaced  by  tha  moments  astlaator,  tha 
empirical  mean  x^  x. 

Uy  -  xB  +  0.45005  ax  (4c) 

Tha  top  sign  Is  valid  for  maxima,  tha  lowur  ml gn  for  minima. 

Tha  momenta  estimators  are  simple  to  compute'  and  have  bean  employed 
by  Gumbel  (1958)  moat  of  tha  time,  probably  because  at  that  tlmo 
the  computer  methods  Were  not  as  widaspraad  as  today. 

b.  Maximum  Likelihood  Estimators 

The  maximum  likelihood  estimators  must  be  determined 
from  the  maximum  likelihood  equations,  which  can  be  readily  derived 
(e.g.  Gumbel  1958),  as  follows 

*  A  AA  * 

1  -  a  x  +  o  a  au(2xa“ax)/N  -  0  (5a) 

1  -  (E  a  ~“*)/N  -  0.  (5b) 

The  mean  value  can  be  taken  from 

u  *  x  -  a/a  (5c) 

which  is  equation  (4)  modified  for  the  maximum  likelihood  estimators. 
It  is  evident  that  no  explicit  solution  can  be  obtained  in  the  form  of 
an  analytical  expression  for  a,  although  equations  5a  and  b  could  be 
combined  and  u  eliminated.  This  would  provide  one  equation,  namely 


1  -  5  ;  +  a  (le  -“V1 


(Exe  "ox)  ■=  0. 
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It  should  be  noted  however,  that  either  we  assume  that  x  can  be  replaced 
by  the  mean  value  xm  or  we  introduce  u  back  via  equation  (5c), 

Then  equation  (5d)  ia  again  one  sejtfiTtion  with  two  parameters  to  be  determined 
Equation  (5d)  can  be  solved  by  iterative  procedures  irrespective  of  our 
choice  for  x.  Electronic  data  processing  has  made  it  possible  today  to 
obtain  solutions  readily.  Equation  (5d)  has  more  than  one  solution, 
however,  and  a  straight  iteration  with  the  moments  estimators  as  initial 
conditions  may  not  always  lead  to  a  unique  solution,  as  It  la  unknown 
whether  the  maximum  likelihood  parameters  are  smaller  or  larger  than  the^ 
moments  estimators.  In  the  case  that  the  extended  forms  with  arbitrary  u 
is  employed,  determination  of  a  would  be  even  more  complex  as  the  u  and  a 
pair  must  be  checked  independently,  a.g.,  by  equations  (5c)  or  (5a).  Than 
again,  the  x  must  be  known,  and  we  can  replace  x  by  the  empirical  mean  from 
the  beginning.  An  independent  check  would  be  the  median  or  percentiles. 

By  and  large,  the  computer  coats  for  maximum  likelihood  approxi¬ 
mations  are  reasonable  if  only  a  few  mamqlms  are  studied.  In  our  case 
with  12  months,  25  altitude  Ivreltg/JtKt  .several  stations  selected  from 
characteriatlc  climatic  regions  coat  reduction  is  a  substantial  contribution, 

c.  Minimum  Variance  Estimators 


Other  deficiencies  such  as  the  determination  of  expected  values 
of  the  estimators  u  and  a  for  statistical  Judgment,  their  probability 
distribution,  the  efficiency  and  other  properties  have  lead  Lleblain  (1954) 
to  investigate  whether  optimum  estimators  can  be  found.  His  approach  is  the 
development  of  an  order-statistics,  largely  for  small  samples,  but 
later  extended  to  larger  samples.  This  order  statietics  provides 
an  unbiased  estimator,  whose  efficiency  can  be  simply  and  accurately 
evaluated.  We  can  write 
m 

u*  «j«jA  (6a) 


B*  -jJk  b^/k  (6b) 

where  a  ■  1/8  and  a^  and  are  weighting  factors  given  in  table 
form  up  to  m  ■  6,  by  Lleblain,  The  s^  is  the  column  sum  of  a  data 
sample  matrix,  broken  into  m  •  k  *■  N  elements. 


Present  availability  of  electronic  data  processing  simplifies  the 
task  of  calculating  u*  and  B*,  and  computations  are  less  costly  than  the 
maximum  likelihood  estimators.  Some  difficulty  arises  in  the  arrangement 
of  an  m  •  k  matrix,  as  prime  numbers  cannot  be  broken  into  m  •  k  factors 
where  m  <  6  and  m  and  k  are  integers.  Furthermore,  the  m  should  always 
be  selected  as  large  as  possible. 


► 


Whan  N  -  (n  •  k  +  in'),  where  a,  k,  in'  i  1,  but  m  and  m'  s  6,  than 

u*  -  (w1  uj  +  w2  uJ)/N  (6c) 

6*  -  <«i  ej  +  w2  eJ)/N  (6d) 

The  w^  -  m  •  k  and  ■  m' ,  with  subacript  1  denoting  the  computation  of  an 

aacimator  from  tha  m  •  k  data  and  subacript  2  tha  one  from  m*. 

Although  tha  procedure  to  break  N  into  m  •  k  +  m*  sections  can  be  com¬ 
puterised  for  electronic  data  processing,  it  may  sometimes  be  an  In¬ 
efficient  part  of  tha  program,  Tha  optimum  choice  of  m  •  k  +  ra'  may 
not  always  be  accomplished.  However,  the  raquiramants  to  establish 
an  m  •  k  data  matrix  lead  to  other  restrictions,  too.  First,  we  need 
the  data  kept  in  sampling  order  for  the  establishment  of  the  m  sections. 
This  eliminates  application  to  any  sample  where  grouping  has  been  made 
or  where  the  sequence  of  sampling  is  unknown.  The  second  problem  is 
the  ra-arrangemant  into  the  data  matrix  form,  as  the  N  may  not  be 
known  a  priori,  especially  whan  tha  extreme  value  sample  must  be  ex¬ 
tracted  from  a  larger  collection  of  data,  whose  period  of  records 
differs  in  length  and  cannot  be  normalised  a  priori.  These  technical 
problems  have  nothing  to  do  with  the  statistical  background  under 
which  Liable in  has  developed  the  solution  of  the  minimum  variance 
unbiased  linear  estimators.  Under  the  goals  set  by  him  these  esti¬ 
mators  are  the  optimum  solution.  Again,  for  a  few  data  samples  with 
small  number  of  observations  these  estimators  may  be  the  ideal  answers, 
especially  when  one  is  interested  in  the  computation  of  the  efficiency 
of  the  estimators.  Although  computation  time  is  reduced,  the  gain  is 
not  very  substantial. 


d)  Modified  Maximum  Likelihood  Estimators 

The  question  arises  whether  approximations  to  the  likelihood 
equations  exist  which  lead  to  simplification,  and  give  an  explicit  answer 
for  a  at  small  computer  costs.  A  positive  answer  to  the  question  can  be 
found  if  we  relate  the  maximum  likelihood  equations  to  the  s  systems  since 
the  F(s)  is  known.  We  replace  therefore 

eau  e-«x  _  a-«t  (?) 

a  x  -  a  u  +  5,  (7a) 

and  ax-z+ax-z,  (7b) 
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(7c) 


Furthermore  we  may  eet  N  ■  £  F(x)  -  1  end  obtain 

a  u  -  in  E  e  “ax  (7c) 

This  leede  from  the  origical  eqn,  5a  to  the  following  equation 

1  -  eij  S  -  a1  E  x  e  ”*  *  0  (8) 

or  [X  -  E  x  e  “*]  •  1.  (8a) 

We  may  replace  x  by  the  empirical  mean  x  ,  The  term  in  the  eunaation 

can  be  computed  by  eubatituting  for  the  population  F(a)  the  empirical 
cumulative  P(x).  Then 

F(x)  ■  exp  {  -  exp(-a))  (8b) 

end  a  »  in  (-  in  F(x)].  (8c) 

Since  for  H  -*■  •  the  F(x)  +  F(a),  the  maximum  likelihood  aetimator  a  end  the 
(approximation  from  equation  8a)  becoma  identical. 

From  the  original  maximum  likelihood  aquation  we  can  further  develop 
a  eacond  aolution  by  multiplication  of  the  maximum  likelihood  aquation 
with  x.  After  aoma  arithmetic  operation  we  derive 

S*  (E  x2  e"a  -  E  x2)  +  2  a  x  -  I  -  0.  (9) 

Thie  ie  a  quadratic  equation  and  haa  two  solutions,  which  lead  to  a 
positive  and  negative  u.  The  anewer  with  the  negative  u  can  be  discarded. 


3.  COMPARISON  OF  THE  ESTIMATORS 

The  first  task  would  be  estimating  the  error  which  is  introduced 
by  replacing  F(x)  by  F(x).  Assume  that  ws  know  x  to  a  degree  x  +  c^, 
which  glvae  an  error  in  s  of  s  +  c|.  Then 

N  e  -  E<x  +  c  )  a  “**  +  'a*  -  E  x  e“*.  (10) 

Cl  X 

Thie  can  be  broken  into 

N  e  -Ex  s’**1  +  cs^  +  E  e  e"^“  +  -Ex  e"“*  <10*> 

“  “ce  * 

If  we  assume  that  e  >1,  since  t(  will  be  a  small  else,  we  obtain 
approximately 


e  -Tee. 
°1  x 
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It  it  tvidenc  that  for  N  -+•  •  the  c  -*■  o  and  e  ->  o,  especially  since 
-e  x  al 

then  a  -  1.  We  further  can  assume  that  c  will  be  positive  and  negative 

random  errors  and  thsn  c  should  be  small.  It  may  suffice  to  test  t  by 

“l  *1 

computations  from  actual  data  on  hand. 

Attention  should  be  further  directed  towards  tha  fact  that  a  itself  is  an 
estimator  and  tha  observations  x  era  attached  with  an  error  c  for  estimation 
of  a.  * 


Comparison  between  ths  maximum  likelihood  estimator  a  and  the  four  other 
estimators  has  been  made  for  2  seasons,  Summer  and  Winter,  for  levels  1 
through  24  km  and  for  4  stations  from  different  climatic  regions  (Albrook, 
Canal  Zone;  Montgomery,  Alabama;  Berlin,  Germany  end  Thule,  Greenland), 

Than  the  percentage  difference  was  computed,  as  the, a  varied  in  sise, 
especially  for  the  density; 

A  -  (Oj  -  a)/a.  (11) 

where  the  repreaunts  the  4  other  estimators  introduced  previously. 

The  mean  (2),  standard  deviation  0^,  and  the  absolute  mean  difference 

| A |  was  then  calculated  for  the  entire  set  of  estimators.  The  result  is 
exhibited  in  Table  1,  It  is  evident  that  the  displays  the  closest 

approximation  of  all  estimators,  as  mnpected,  with  average  deviations 
less  than  3  percent. 

_  Although  the  minimum  variance  estimators  a*  discloses  a  similar  maan 
(A)  value,  it  can  be  seen  that  the  mean  absolute  deviation  (in  percent) 
is  by  far  higher  and  the  standard  deviation  each  of  the  96  estimators 
(Summer  and  Winter  each)  is  by  far  greater  than  for  the  a^.  This  is  no 
surprise,  and  illustrates  the  applicability  of  the  solution  by  equation 
(8a)  for  practical  purposes.  The  computer  time  to  calculate  was  only 

a  minute  fraction  of  the  time  for  a  and  considerably  less  than  for  a*. 

It  must  be  further  called  to  attention  that  although  the  computation 
is  based  upon  an  order  term,  the  sample  sequence  need  not  be  known  a 
priori.  The  method  can  even  be  applied  to  grouped  data,  reducing  merely  the 
number  of  summations  for  E  x  e  ,  It  is  understood  that  the  x  must  then 
be  replaced  by  the  upper  boundary  value,  as  we  are  dealing  with  cumulative 
distribution  in  setting  F(z)  -  F(x). 

Tables  2  through  4  exhibit  the  results  for  a,  and  u.  for  the  station 
Montgomery  as  an  example  that  the  findings  for  J  ^  the  individual 

stations  resemble  the  summary.  The  survey  was  also  made  for  the  absolute 
deviation,  A  ■  -  a  (lie) 

but  the  outcome  is  similar  to  the  one  presented  and  may  be  omitted. 
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Vinter 


TABLE  1 

Comparlaon  with  Maximum  Likelihood  for  a 
(1  through  24  km,  Summary  4  beat  Iona) 


Penalty 


Eqn.  8e  -.6  4.3 

Eqn.  9  -5.6  4.1 

Horn .  7.7  6.8 

Min.  Vei  -a. 7  9.8 

Eqn.  £*  -G.8  2.6 

Eqn.  9  .5.6  >.4 


|Z| 

- 

eg 

|A|  2 

9.2* 

8.6* 

10.9* 

9.8#  15.0?  15.2* 

7-91 

11.8 

10.7 

1.9 

9.2 

9.2  4.9 

15.4 

4.5 

5.5 

-1.5 

2.0 

2.2  -1.1 

2.3 

4.1 

7.2 

*1.4 

2.1 

2.3  -1.1 

2.4 

6.8 

8.9 

15.8 

9.7 

16.3  U-* 

9.2 

9.8 

9.0 

1.0 

10.3 

8.3  0.0 

9.2 

2.6 

5.4 

-.7 

2.8 

2.6  -.9 

2.6 

5.4 

6.0 

•  .6 

5.0 

2.7  ••2 

3.9 

Temperature 
I  c A  I  IAI 
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TABU  2 


Comparison  with  Maximum  Likelihood 
Hind  Spaod  < 1  through  2k  km  Altitudo) 

A  in  It 


A 

0 

‘a 

1AI 

A 

u 

‘A 

»T| 

Mom 

4.4* 

0.5 * 

-<* 

.0.7* 

Min.  Var. 

13  A 

1.6 

1.7 

Eqn.  8a 

2.2 

'  fl  ■ 

.3 

♦3 

Eqn.  9 

D 

1.8 

.3 

1.0 

Mom 

8.4* 

8.6* 

9.4* 

.6* 

•T* 

.7* 

Min.  Var. 

-3.5 

10.3 

9.1 

•.0 

.9 

•7 

Eqn.  8a 

-1.8 

2.7 

-.1 

•3 

•2 

Eqn.  9 

-4.4 

4.2 

-.6 

.8 

.6 

aummer  a: 

Range  .13 

through  .34 

us  Range  7>1  through 

32.3  m/aae 

winter  a: 

Range  .10 

through  .79 

u;  Range  9.8  through 

77.5  m/aae 
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Vinter 


TABU  3 


Coaperlion  with  Maxima  Likelihood 
Teaperature  (1  through  24  ka  altitude) 

4  in  * 


4 

a 

ff4 

!A| 

4 

u 

v_ 

I4| 

HO® 

ia,2* 

0.1* 

12.6* 

.03* 

.02* 

.03* 

Min.  Tar. 

•  .8 

10.3 

7.9 

.01 

.03 

.02 

Bqn. 

-1.4 

2.1 

2.2 

-.00 

.01 

.01 

Bqn. 

-1.4 

2.1 

2.3 

.00 

.01 

.01 

Mon 

14.2* 

7.1* 

14.6* 

.06* 

.03* 

.06* 

Min.  Var. 

-5.9 

6.9 

8.3 

-.01 

.03 

.03 

Eqn. 

•*1.4 

1.2 

1.6 

-.01 

.01 

.01 

Eqn. 

■1.5 

1.3 

1.7 

-.01 

.01 

.01 

suaaer  a:  Range  .47  through  1.27  «*j  Range  J06.9  through  200. 7°K 

winter  cm  Range  .33  through  .70  ut  Range  294, 3  through  211. 9°K 


TABU  4 


Comparison  with  Maximum  Likelihood 
Danalty  ( 1  through  24  km  Altlfcuda) 

A  in  * 


'  . A 

a 

°A 

1*1 

A 

u 

°A 

1*1 

Mom 

5.6* 

11.4* 

9.7* 

.02* 

.04* 

.03* 

Min.  Var. 

-  .2 

8.9 

7.0 

.01 

.04 

•03 

Bqn. 

-1.5 

2.6 

2.3 

-.00 

.01 

.01 

Bqn. 

-1.7 

2.1 

2.4 

-.00 

.01 

.01 

Mom 

16.8* 

10.2* 

16.8* 

.08* 

.05* 

.08* 

Min.  Var. 

2.7 

13.1 

10.5 

.02 

.07 

.06 

Bqn. 

-  .4 

3-0 

2.4 

-.00 

.01 

.01 

Bqn. 

-  .4 

mm 

2.5 

-.00 

.02 

.01 

simmer  a:  Range  ,12  through  2.20  u;  Range  1.195  through  .059  g/ca* 

winter  oc.  Range  .04}  through  1.80  ut  Range  1.309  through  .055  g /cm* 
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4,  COMPARISON  OF  THE  OBSERVED  AMD  ANALYTICAL  FREQUENCY 

The  discussion  of  tha  axtraftt  vsluss  would  bs  incomplete  if  no 
mentioning  wars  mads  of  tha  suitability  of  tht  distributions  to  upper 
sir  data.  The  bast  method  of  parameter  estimation  is  to  no  avail  if  tha 
distribution  model  doss  not  satisfy  tha  observations .  It  was  not  intandad 
to  have  a  one-model  comparison.  Hence  tha  Pieher-Tlppett  II  distribution 
was  also,  employed  for  Judgment.  We  have  ■■ 


F(a)  -  exp  <-s“Y) 

Q2) 

with 

a  -  x/S. 

(12a) 

Tha  momenti 

■  fit  ia 

/.  \2 

+  1  -  P  (1  “  2/y)  /r2  (1  -  l/y) 

i  N 

(13a) 

and 

8  -  xB/r  (l  -  l/y) . 

(13b) 

Tha  Maximum  likelihood  fir  renders  the  equations 

fa  t.  *»■*,- ALflO  ■  0 

Y 

(13c) 

a  -  t  In  .  ♦  * 1  *'Y  *  -  0. 

Y  E  x 

(13d) 

It  ie  evident  that  the  moments  estimators  are  not  simpler  to  compute 
than  tha  maximum  likelihood  estimators  as  the  y  cannot  be  expressed  in 
explicit  form  from  the  gamma  function  except  by  approximation.  One 
can  therefore  calculate  the  maximum  likelihood  estimators  just  as  well. 
The  solution  must  be  found  by  iterative  procedures. 

After  determination  of  the  maximum  likelihood  estimators  for  the 
Gumbal  distribution  (from  equation  8a)  and  fox  the  Fisher-Tlppett  II 
curve  the  analytical  F(x)  was  computed  for  both  systems.  The  analytical 
and  empirical  cumulative  frequencies  were  compared  and  the  deviations 
checked  by  the  Kolmogorov-Smirnov  significance  test: 


Dm  -  nut  I  F(x)  -  F(x0)|  (14) 

where  F(x)  la  Che  analytical  and  F(xq)  the  empirical  value  for  thraahold 

x,  It  was  decided  to  teat  on  the  95%  aignif leant  level.  Since  moat 
of  the  judgment  by  the  Kolmogorov-Smirnov  teat  for  email  N  la  aomehow 
too  optimistic,  the  modification  by  Lilllefore  (1967)  hae  been  employed, 
which  laada  to  elgnlflcanca  at  aome  lower  maximum  D^.  The  teat  criterion 

D0  waa  taken  from  hie  tablet.  The  deviation  was  conaldared  significant 

when  Dm  £  D^. 

The  lnveatlgation  for  Montgomery  aa  tho  teat  etation  disclos  :hat 
Gumbel'a  law  and  Fisher-Tippett  II  are  both  good  approximations  f  che 
wind  apaed  at  altitude  levala.  Only  at  one  altitude  level  the  significance 
threshold  waa  exceeded.  Thla  occurred  for  every  model  which  waa  tested, 
not  only  one  method  and  la  not  critical.  In  a  significance  teat  with 
48  samplaa  at  the  95%  level  2  samples  may  exceed  the  95%  criterion. 

A  typical  example  (Fig.  1)  waa  then  selected  with  the  6  km  altitude 
and  winter  conditions  at  Montgomery,  Alabama.  Although  in  this  case  no 
significant  deviation  could  be  noted  for  the  Fieher~Tippatt  II,  the 
fitting  of  the  observed  data  appears  batter  for  Gumbel'a  law.  The  average 
absolute  daviatlona  from  the  observed  data  are  highest  for  the  Fisher* 
Tippett  II. 

The  temperature  extremes  were  studied  next.  This  time  3  altitude 
levels  displayed  some  exceedance  of  the  significance  threshold  for 
Gumbel'a  law.  This  number  may  be  still  considered  in  line  with  the 
selected  threshold  of  significance,  especially  in  comparison  with  the 
Fisher-Tippett  II  law.  Only  18  samples  out  of  48  showed  no  significant 
deviation  for  the  latter.  Consequently  we  would  conclude  that  Gumbel'a 
law  seems  better  suited  to  represent  temperature  extreme  values  of 
upper  air  data.  If  the  same  number  of  deviations  for  both  methods  or 
the  upper  air  wind  spaed  had  occurred  one  would  probably  Interpret  the 
result  as  an  effect  of  missing  observations,  or  other  data  related 
causes }  especially  high  wind  speeds  may  be  missing.  This  explanation 
fails  for  the  temperature,  however. 

A  typical  example  is  given  in  Figure  2  for  Montgomery  at  the  8 
km  level  in  summer,  when  tho  Fisher-Tippett  II  comparison  provided  no 
significant  deviation.  Again,  the  Fisher-Tippett  II  curve  exhibits  the 
largest  deviations. 

Finally  upper  air  density  extremes  were  considered.  This  time  the 
number  of  olgnificant  deviations  from  the  observed  cumulative  frequency 
was  higher  for  the  Gumbel  law  with  eight.  This  is  higher  than  one  would 
normally  expect  at  this  significance  level.  Had  we  employed  the  threshold 
D  from  the  unmodified  Kolmogorov  test  no  significance  would  have  turned  up. 
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Figure  1 


MONTGOMERY,  WIND  EXTREMES 
ALTITUDE  6  KM,  WINTER 
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MONTGOMERY.  TEMPERATURE  EXTREMES 
ALTITUDE  8KM,  SUMMER 
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Figure  3 


DENSITY  (Q/CM3) 
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The  Fishsr-Tippett  II  behaved  similarly  to  the  outcome  for  the 
temperature,  only  18  aeaplea  ahowed  no  eltnlflcance,  although  eamplee 
with  exceedance  for  the  temperature  did  not  automatically  prove  significant 
for  the  density.  Therefore  Gumbel's  law  aeema  to  fit  the  obeervational 
data  better  again. 

The  higher  number  of  significant  deviations  for  the  density  may  have 
some  other  explanation.  The  density  is  an  element  derived  by  the  gas  law 
from  pressure  end  temperature.  The  density  extremes  may  therefore  come 
from  a  collective  of  a  bivariate  distribution.  This  modification  has 
only  very  recently  been  studied  by  Campbell  and  Taokos  (1972).  Their 
findings  prove  that  correlated  normal  maxima  ere  asymptotically  uncorrelated. 
Thus  density  extremes  would  come  from  uncorrelatad  pressure  end  temperature 
data  even  with  the  existence  of  correlation  between  temperature  and  praasure. 
This  modification  may  affect  the  distribution.  Further  investigates 
seem  appropriate  before  final  conclusions  can  be  drawn.  9- 

5.  CONCLUSIONS:  It  has  baan  demonstrated  in  the  previous  sections 
that  the  maximum  livelihood  estimators  can  be  approximated  by  a  slight 
modification  of  the  likelihood  equations  employing  the  analytical 
cummulative  distribution  function.  This  presents  an  explicit  solution 
for  the  estimator  a  and  thus  reduces  the  time  of  computation  considerably. 

The  method  has  the  advantage  that  data  need  not  be  given  in  sequence 
of  data  sampling,  which  is  required  for  Liebleln's  minimum  variance 
linear  order  statistics  estimator.  Establishment  of  an  m  •  k  data  matrix 
can  also  be  waived.  Comparison  of  the  two  approximate  estimators  with 
the  maximum  likelihood  estimator  revealed  that  especially  the  first 
suggested  method  given  excellent  agreement  with  the  correct  solution. 

The  new  estimators  have  the  properties  of  maximum  likelihood  estimators. 

A  final  comparison  of  the  analytical  cummulative  frequency  with  the 
observed  one  was  performed.  In  this  study  the  analytical  curve  from  the 
Fiaher-Tippett  II  extreme  value  distributions  was  added.  Gumbel’s  law 
and  the  Fisher-Tippett  II  model  appear  equivalent  for  the  wind  speed  at 
the  tested  significance  level,  Montgomery  data  with  2  seasons  and  24 
altitude  levels  served  as  the  checking  station. 

The  Gumbel  distribution  proves  significantly  better  suited  for  upper 
air  temperature  and  density.  Even  for  wind  speed  the  differences  between 
analytical  and  observed  distributions  ia  smaller  for  Gumbel’s  law. 

With  the  availability  of  an  explicit  solution  for  the  maximum  likelihood 
estimators  it  should  not  be  difficult  to  apply  them  to  a  large  number  of 
sampler,  as  even  then  coats  for  electron c  data  processing  stay  within 
reasonable  limits,  which  would  not  be  true  for  iteration  procedures. 
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ABSTRACT,  The  aim  of  this  paper  is  to  develop  statlatlcel  models 
to  forecast  short-path  oblique  incidence  (01)  high  frequency  (HF) 
information  up  to  a  certain  time  in  advance,  by  utilising  the  observed 
vertical  incidence  data  over  typical  field  army  distances. 

It  la  shown  that  there  is  a  strong  linear  dependence  between  the 
oblique  and  vertical  incidence  ionospheric  soundings.  Linear  regression 
models  have  been  developed  to  astimata  the  oblique  incidence  soundings  from 
observed  vertical  incidence  recordinga. 

It  la  further  shown  that  ionospheric  data  of  thla  type  is  a  non¬ 
stationary  stochastic  realisation,  A  procedure  is  presented  in  modeling 
such  information  for  the  purpoae  of  forecasting  one,  tun,  three,  .  .  ., 
k  time  slots  ahead  over  a  given  path.  Autoregressive  and  moving  average 
forecasting  models  have  been  formulated  for  the  60  km  path  from  Fort 
Monmouth  to  Fort  Dlx.  Confidence  bounds  have  been  obtained  for  both 
the  linear  regression  models  and  the  time  series  analysis  of  the  ionospheric 
data. 


1,  INTRODUCTION.  The  field  army  employs  many  means  of  communication, 
each  of  which  is  tailored  to  fit  a  particular  requirement.  Specifically, 

H.F,  communications  provide  systems  not  specifically  limited  by  llne-of-slght 
extended  distance,  or  Intervening  terrain  obstacles.  However,  size  and 
weight  of  tactical  communications  equipment  must  be  kept  to  a  minimum, 
and  the  tactical  communicator  must  contend  with  a  relatively  low  power 
transmitter,  and  physically  small,  inefficient  low  gain  antennas. 

Ionospheric  disturbances,  both  natural  and  man-made,  further  complicate 
hie  problems.  The  H,  F,  communicator  haA,  as  his  only  propagation  aid, 
the  monthly  predications  for  undisturbed  conditions  prepared  three 
months  in  advance  by  the  Department  of  Commerce,  and  distributed  by  the 
US  Army  Strategic  Communications  Command,  While  valid  for  long  range 
planning,  they  do  not  account  for  diurnal  variations  or  disturbed 
ionospheric  conditions  which  may  harass  him.  As  a  result,  the  U,  S, 

Army  Electronics  Command  is  developing  a  system  to  provide  tactical 
communicators  with  propagation  predictions,  in  near  real-time,  and 
prepared  specifically  for  the  Army  area  of  interest. 
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The  aim  of  this  paper ,  therefore,  ie  two-fold: 

a.  to  introduce  a  new  statistical  concept  to  the  estimation  of  oblique 
Incidence  soundings,  either  knowing  or  being  able  to  predict  the  vertical 
incidence  sounding,  and 

b,  to  develop  statistical  models  to  forecast  either  the  oblique 

or  vertical  incidence  soundings  over  specific  paths,  or  at  specific 
terminals,  one,  two,  three,  .  .  . ,  k  time  slots  ahead,  beginning  with 
a  certain  origin.  ,  . 

With  respect  to  the  first  objective,  the  widely  accepted  approach 
to  the  subject  area  waa  to  utilise  the  secant  law  to  estimate  the 
oblique  Incidence  sounding.  Simply  stated,  it  1st 

Y  ■  X  secant  <p  where 

V  «  equivalent  oblique  incidence  frequency 
X  ■  observed  vertical  incidence  frequency 
<t'  -  angle  between  the  oblique  ray  path  and  the  normal  to  the 
ionosphere  at  tha  path  mid-point 

This  la  Snell's  lew  used  under  the  assumptions  of  constant  ionospheric 
layer  height,  no  collisions,  no  magnetic  field  affects,  and  a  spectral 
reflection  at  the  path  mid-point.  If  one  assumes  a  curved  ionosphere 
(for  longer  paths),  however,  a  modification  to  the  above  equation,  depending 
upon  the  electron  density  profile,  is  of  the  form: 

Y  ■  k  X  secant';  where  a  practical  range  of  k  is: 

1.00  a  k  i  1,30 

In  view  of  the  poor  results  obtained  by  Krause,  et  al,  [7],  using 
secant  q* ,  the  probable  need  for  mid-point  data,  and  the  dependence  on 
electron  profile  density,  a  more  practical  approach  for  relating  VI  and 
01  data  was  developed,  using  regression  techniques.  Functional 
relationships  were  derived  for  each  experiment  as  a  prelude  to  the 
forecasting  problem. 

In  Section  3,  we  shall  give  the  regression  models  for  the  overall  mean 
and  the  6th  day  measurements  including  reciprocal  path  data  for  the 
60  Km,  200  Km,  and  500  Km  paths.  The  confidence  Intervale  of  these  linear 
models  are  also  given. 

We  have  ehovn  that  there  is  a  strong  linear  dependence  between  oblique 
end  vertical  incidence  noundinge  et  all  the  paths  investigated.  The  regres¬ 
sion  models  developed  show  that  one  can  accurately  estimate  the  equivalent 
oblique  incidence  eounding  for  a  given  value  of  the  vertical  incidence 
date.  What  remains  la  the  need  for  a  model  that,  under  certain  realistic 
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condition*,  will  enable  one  to  foracaat  tha  vartical  incidance  ionospheric 
sounding*  and,  from  the  foracaat,  to  estimate  the  corresponding  01 
sounding.  This  is  the  aim  of  tha  second  part  of  our  paper. 

It  is  shown  that  for  the  60  Km  experiment,  both  the  oblique 
and  vertical  incidenca  recording*  are  non-stationary  stochastic 
realisations.  That  ia,  they  fora  a  discrete  time  aeries  which 
is  not  in  statistical  equilibrium.  He  propose  a  procedure  to 
handle  this  type  of  information  and  to  investigate  the  possibility 
of  characterising  our  data  with  an  autoregressive  process,  a  moving 
average  model,  or  a  mixture  of  autoregressive-moving  average  processes. 

In  Section  4,  we  present  a  systematic  presentation  of  analysing 
ionospheric  soundings  for  the  purpose  of  forecasting.  An  autoregressive 
model  has  been  developed  for  the  discrete  realisation  representing 
the  6th  day  observed  oblique  incidence  critical  frequencies  for  the 
60  Km  path,  Fort  Monmouth,  N,  J,  -  Fort  Dix,  N,  J,,  in  Section  5,  The 
complete  procedure  of  fitting  such  a  model  is  given,  along  with  ite 
confidence  Intervals.  In  Section  6,  we  develop  a  moving  average 
model  that  characterises  tha  behavior  of  the  overall  vartical 
incidence  soundlnga  for  the  60  Km  experiment.  Forecasting  models 
for  tha  200  Km  and  500  Km  patha  from  Fort  Monmouth,  N.  J.,  to 
Aberdeen  Proving  Ground,  Md, ,  and  from  Fort  Monmouth,  N.  J.,  to 
Camp  Drum,  N.  V. ,  raapectively,  are  presently  being  developed. 


The  remainder  of  this  paper  wa*  reproduced  photographically  from  the 
manuscript  submitted  by  the  author. 
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2.  DESIGN  OF  THE  EXPERIMENT. 

To  accomplish  the  task  of  developing  both  a  functional 
between  01  and  VI  maxlatm  bbserwefl  frequencies  (MOP)  and  a  forecast irjj  .v/i*L, 
the  Conmunicat ions/ADP  Iabomtory,  of  the  U.  S.  Army  Electronics  Ccsra tA,  rAu 
been  Involved  In  an  extensive  collection  of  VI  and  short-path  01  low-cpir. eric- 
data  at  three  different  distances  with  Fort  Monmouth,  N.  J.,  an  the  bee* 
station.  Experimentation  was  performed  In  the  2-16  MHz  range,  using  two 
lonosendes,  one  as  a  fixed  terminal  and  the  other  as  a  mobile  terminal,  as 
shown  in  Figure  2.1.  Die  mobile  terminal  was  situated  at  Fort  Dix,  21.  J., 
establishing  a  60  KM  path;  at  Aberdeen  Proving  Ground,  Ml.,  to  establish  a 
200  Km  path;  at  Camp  Drum,  N.  Y. ,  to  establish  a  nominal  500  Kta  path  (Figure 
2.2).  These  particular  distances  were  chosen  to  fall  within  the  idealized 
300  x  300  kilometers  tactical  Field  Army  area  of  responsibility.  The  nominal 
500  Km  path  (actually  41+0  Km)  Is  representative  of  the  diagonal  or  largest 
internal,  ccnmunlcatlons  path  within  the  area  of  responsibility. 

Each  terminal  made  scheduled  soundings  every  ten  minutes  for  9,  19,  and 
13-day  experiments,  respectively.  While  the  fixed  terminal  was  transmitting 
and  receiving  its  own  signal,  the  mobile  terminal  would  simultaneously 
receive  the  same  transmission;  likewise  for  the  mobile  with  respect  to  the 
fixed  terminal  (Figures  2.1  and  2.2).  Both  ionosondes  were  synchronized  to 
the  WWV  (HF),  (National  Bureau  of  Standards)  time  standard  so  that  the 
"remote"  sounder  scans  would  be  precise  with  the  Fort  Monmouth  terminal.  The 
number  of  days  each  experiment  was  performed  has  no  significance  with  respect 
to  the  results  obtained,  but  was  a  matter  of  funding,  Hie  basic  instruments 
used  were  two  Granger  Associates  Model  3905-5  Ionospheric  Sounders,  matched 
with  wide  response  delta  antennas. 

The  frequency  range  of  the  Ionosondes  was  limited  from  2-16  MHz,  in  three 
octaves,  with  1+00  discrete  frequency  channels  per  octave.  Transmissions 
consisted  of  successively  "stepping'1  through  the  channels  of  each  octave  with 
a  pulse  width  of  100  micro-seconds  to  maximize  the  Bystem  sensitivity.  The 
data  is  a  recording  of  the  time  delay  from  lonosonde  to  ionospheric  reflecting 
layer  and  return.  Time  delay  Is  a  measure  of  the  virtual  height  of  reflection 
from  the  layer.  The  trace  of  the  returned  pulse  on  a  scale  of  frequency 
versus  time  delay  (virtual  height)  is  the  lonogram  record,  Figure  2.3.  Iono- 
grara  records  of  the  data  were  taken  on  35ma  film  at  Fort  Monmouth  and  on  light 
sensitive  oscillograph  paper  at  the  remote  terminals.  After  collection  and 
development,  the  ionograms  were  scaled  for  the  extraordinary  critical 
frequencies,  fxF-,  as  shown  in  Figure  2.3.  The  fxFt  data  was  then  compiled 
for  computer  analysis  and  for  comparison  between  the  observed  VI  and  observed 
01  critical  frequencies. 

The  experiment  results  were  dependent  upon  ionospheric  conditions  and 
man-made  noise.  Conditions  were  characterized  by  the  Space  Disturbance  Fore¬ 
cast  Center,  ITS,  Boulder,  Colorado,  as  generally  undisturbed,,  but  some  inter¬ 
ference  occurred.  Seme  data  (ionograms)  were  unreadable  due  to  man-made 
noise,  solar  and  geomagnetic  activity.  For  those  few  records  which  were 
unreadable  (though  signal  was  detected),  simulated  data  was  prepared.  The 
occurrence  of  obscured  data  was  neglibable  over  the  experiments. 
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FIELD  TESTS  FOR  A  NEAR -REAL  TIME 
IONOSPHERIC  FORCASTING  SCHEME 


Best  Available 


3.  REGRESSION  MODEL 


It  has  been  previously  argued  that  the  secant  law  approach  for  estimating 
short-oblique  incidence  HP  information  by  utilising  observed  vertical 
incidence  data  does  not  take  into  consideration  the  stochastic  behavior  of 
the  problem.  That  is,  the  secant  <p  approach  is  a  deterministic  characteriza¬ 
tion  of  the  experiment.  The  complexity  of  such  experiments  is  subject  to 
human  error,  ionospheric  changes,  assumptions  of  constant  layer  height,  of  no 
collisions,  and  of  ho  magnetic  effects,  among  others.  It  Is  quite  unrealistic,, 
therefore,  to  consider  such  a  phenomenon  from  a  deterministic  point  of  view. 

It  has  been  shown  (see  Table  3*1  that  there  exists  a  strong  linear  depen¬ 
dence  between  the  oblique  incidence  and  vertical  incidence  soundings  for  all 
the  distances  investigated  in  the  experiment,  that  is,  60  Kin,  200  Km,  and 
500  Km. 

Thus,  in  view  of  the  strong  linear  dependence  shewn  by  the  oblique  and 
vertical  Incidence  ionospheric  soundings,  we  formulated  pairs  of  reciprocal 
linear  regression  models  for  the  overall  avenges  and  for  the  6th  day  measure¬ 
ments  for  the  three  paths.  These  models  are  shown  in  Table  3*2,  where  X  nnd 
Y  represent  the  vertical  and  oblique  incidence  ionospheric  data. 

The  usefulness  of  such  models  is  quite  clear.  That  is,  at  a  given 
distance,  if  one  is  willing  to  assume  similar  ionospheric  conditions,  seasonal 
variations,  and  terrain,  a  very  good  estimate  of  the  oblique  incidence 
sounding  can  be  obtained,  by  knowing  the  vertical  incidence  sounding. 

Confidence  limita  were  obtained  for  each  of  the  models  shown  in  Table  3.2. 
More  specifically,  95 t  confidence  intervals  have  bean  computed  and  plotted  for 
the  overall  averages  and  the  6th  day  measurements  for  each  of  the  paths 
Gtudied.  Figures  3.3  through  3.1U  show  the  formulated  regression  models  with 
their  computed  confidence  limits. 

For  a  complete  study  of  the  regression  analysis  modeling  01'  ionospheric 
data,  refer  to  D'Accardl  and  Tsokos  [6], 
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1)..  TIME  SERIES  MODELING:  IDENTIFYING  THE  STOCHASTIC  REALIZATION 


In  a  given  physical  situation  such  as ' in  the  ionospheric  sounder  project, 
we  have  available  a  tine  series,  say,  x1;  x* , x*,  of  n  observations. 

Our  aim  is  to  obtain  a  suitable  difference  equation  or  model,  that  will 
accurately  represent  the  true  underlying  process  which  generated  the  iono¬ 
spheric  soundings,  xt ,  t*l,2, . . .,n.  First,  we  must  identify  whether  the 
series  xt  exhibits  stationary  or  non- stationary  properties.  That  is,  when 
we  speak  of  stationary  time  series,  we  imply  that  the  statistical  properties 
of  the  series  are  independent  of  absolute  time.  A  graphical  representation 
of  the  ionospheric  soundings  would  be  of  some  aid  in  exercising  Judgment 
about  the  behavior  of  the  data.  Of  greater  lipportanee  is  the  sample  autocor¬ 
relation  function  given  by: 

rxx(k)  =-  ,  It  *  0,1,2,  ...,n-l 

where  cxx (k)  is  the  sample  autocovariance  function  defined  by: 

n-k  _  _ 

cXJI(k)  *  i  ?,  (xt  -  x)  (xt+k-x),  to- 0,1,2,  ...n-l, 
n  ttl 

of  the  observed  soundings.  If  the  ionospheric  soundings  are  stationary,  the 
sample  autocorrelation  function  would  exhibit  fairly  rapid  dampening. 
Furthermore,  one  can  apply  various  statistical  tests  to  check  for  non¬ 
stationary  properties.  Wo  have  uoed  Kendall’s  tau  test  [*)• 

If  the  ionospheric  soundings  were  not  in  statistical  equilibrium 
(stationary),  then  we  can  filter  out  the  non- stationary  components  by  using 
various  difference  filters.  In  all  cases,  that  is,  the  CO  km,  200  ton,  and 
',>00  km  experiments,  the  resulting  data  was  shown  to  contain  non- stationary 
components. 

A  general  difference  filter  is  given  by; 

J\  *  (l-n)d  x*  , 

where  B  is  a  shift  operator  and  d  is  the  order  of  the  filter.  When  tl-0,  this 
will  indicate  that  the  ionospheric  data  i.v  stationary;  d~l,  will  indicate 
that  a  first  difference  filter  is  necessary  to  filter  the  original  series, 
and  so  on.  For  the  ionor.pharic  soundings  information,  we  used  filters  up  to 
d  «  2. 


The  procedure  to  determine  the  proper  value  for  d  is  to  compute  the 
first  differences  of  the  original  data,x, ,  t&l,2,  .  ,.,n.  That  is,  wo  procen; 
xt  through  a  firBt  difference  filter: 

yt  =  (l-B)  xt  =  xt  -  x,_j  , 

which  will  have  (n-l)  observations,  and  then  through  a  second  difference 
filter: 

wt  =  ( l-B )f  t  *  xt  -  2xt_i  ►  xt.j 

which  will  liave  (n-2)  observations.  For  the  original  soundings,  xt ,  and  the 
filtered  yt  and  w, ,  we  calculate  the  sample  autocorrelation  function  and 
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Kendall's  tau  r51:.  by  observing  the  cample  autocorrelation  function  of  the 
original  series, the  filtered  soundings,  and  the  results  of  the  trend  test,  we 
can  infer  a  suitable  value  for  d,  that  is,  the  degree  of  "differencing" 
necessary  to  induce  the  sample  autocorrelation  function  to  dampen  out  fairly 
rapidly  and  cause  Kendall’s  tau  test  not  be  significant. 

For  predicting  or  forecasting  oblique  incidence  data  for  one  or  more  time 
slots  in  advance,  the  initial  step  i6  to  determine  the  particular  process 
that  characterizes  our  data.  There  are  three  basic  models  that  are  candi¬ 
dates  fot  this  purpose: 

a.  The  Autoregressive  Process 

b.  The  Moving  Average  Procese 

o.  The  Mixed  Autoregressive-Moving  Average  Process. 

A  discrete  m-otxter  autoregressive  model  is  of  the  form: 

Xt-n  =  +  Oa(xt_a-u)  +  ...  +  a,  (xt_,-u)  +Zt  (4.1) 

where  xt  is  the  autoregressive  series  which  is  being  generated  by  the  series 
Zt «  a  purely  random  process,  dj ,  0g,  ...^0,  are  the  parameters  of  the  non- 
ordered  process,  and  M  is  the  expected  value  of  the  series.  Such  a  process 
assumes  that  the  current  value  xt  of  the  soundings  has  resulted  from  a  linear 
sum  of  past  values  of  the  series.  Such  a  process  assumes  that  the  current 
value,  xt ,  of  the  soundings  has  resulted  from  a  linear  sum  of  past  values  of 
the  series,  together  with  an  independent  error  term,  Zt ,  not  connected  with 
the  past. 

A  discrete  q-order  moving  average  process  is  given  by: 

xt  -  U  =  .'i  -  *  •••  “  Pq^'t-q  •  (  **2) 

Till::  process  is  n  weighted  sum  of  a  random  series,  Zt .  Each  realization 
(oblique  incidence  sounding,  x, )  is  made  linearly  dependent  on  a  Zt  and  on 
one  or  more  previous  Z'c.  Also,  u  is  the  expected  value  of  xt ,  and  ,  pa, 

Sq  are  the  parameters  of  the  model. 

The  mixed  model  consists  of  the  autoregressive  and  moving  average  model 
where  m  is  independent  of  q. 

We  shall  discuss  in  some  detail  a  procedural  approach  in  fitting  an 
autoregressive  model  to  the  ionospheric  data  series.  A  similar  approach  can 
be  followed  to  formulate  the  procedure  for  the  moving  average  and  mixed 
process  with  minor  changes. 

4 . 1  The  Autoregressive  process 


The  autoregressive  model  previously  defined  can  be  adapted  to 
represent  the  characterization  of  ionospheric  data  for  the  purpose  of  fore¬ 
casting.  Discussing  the  theory  of  the  general  mth  order  model  is  quite 
complicated  and,  therefore,  we  shall  first  give  a  brief  discussion  of  the 
second  order  model  which  is  quite  useful  in  many  physical  situations. 

The  second  order  discrete  autoregressive  process  may  be  written  as: 
x,  -  pi  =  oq  (xt_t  -pi)  +  ^(xt.g-p)  +  Z,  .  (4.3) 
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If,  at  the  initial  stage,  it  was  necessary  to  filter  the  ionospheric  data  to 
have  the  information  in  statistical  equilibrium,  then  we  must  place  certain 
restrictions  on  estimating  the  parameters  of  the  model  to  make  sure  that  our 
series  remains  stationary.  To  obtain  these  restrictions  on  the  parameters, 
we  use  the  concept  of  -transforms,  to  obtain  the  characteristic 

equation  of  the  process.  Solving  the  characteristic  equation,  we  can  place 
conditions  on  its  roots  so  that  the  fitted  model  will  not  violate  the  assump¬ 
tion  of  stationarity , 

The -transform  of  equation  (4.3)  is  given  by: 

(l-aiT?"l-a,’3f-3)  (xt-u)  z.  > 

and  its  transfer  function  H (3?"1 )  is  given  by: 

scar1 )  = 


Thus,  the  characteristic  equation  of  the  second-order  autoregreRBi vr  model  is 


-  a,|r  -  ftg  =  0 


whose  roots  are  given  by: 


Ci  •  <*i  +/<hU+l*h  and  Ca 


In  order  for  the  second-order  model  to  be  stationary,  we  must  restrict  the 
estimates  of  the  parameters  otj  and  a,  so  that  the  roots  of  equation  (4.4) 
will  bo  contained  within  a  unit  circle,  that  is,  |  Cj  and  |  must  be  less 
than  one.  This  is  equivalent  to  having  a-  and  Og  lie  in  a  triangular  region 
formed  by  di+Oa<  1,  a--Ui<  1,  and  ^1.  For  additional  details  see 

m.  [?]»  and  '31. 


A  similar  approach  can  be  carried  out  by  considering  models  of  higher 
order*.  They  -transform  of  the  order  autoregressive  (4.1)  process  is 
given  by: 

(l-<^y*,,-aa‘Z"a . O.’Z-*)  U»-U)  =  Zt  .  Ot. 5) 

The  transfer  function  of  (4.5)  is  of  the  form: 


Hfjrt*  7 ^ 


The  characteristic  equation  of  the  m  order  process  is  given  by: 


-oLar-3- 


-  «■  -  0 


Tlius,  for  the  general  finite  autoregressive  model  to  be  in  statistical 
equilibrium,  we  must  estimate  the  parameters  of  the  process  so  that  the 
roots  of  equation  (4.7)  must  lie  within  a  unit  circle. 


4.2  The  Fitting  Procedure 


The  initial  stage  in  developing  any  one  of  the  three  models  under 
consideration  usually  involves  deciding  the  order,  m,  of  the  model,  and  then, 
giVon  m,  estimating  the  parameters,  pi,  ax ,  On,  . ..,  <\  . 


The  criterion  for  selecting  the  best  order  which  characterizes  the 
given  series  is  based  upon  the  residual  variance.  We  proceed  by  estimating 
the  parameters  of  the  model  for  different  orders,  and  then  the  residual  vari¬ 
ances  are  computed  and  plotted,  again,  t  the  -rder  of  the  process:.  The  minimum 
residual  variance  will  correspond  to  the  order  of  the  model  which  best 
describes  the  ionospheric  soundings.  'Taut,  for  the  autoregressive  process, 
it  is  necessary  to  first  study  the  estimation  of  the  parameters  of  the  model. 


To  estimate  the  parameters  of  this  process,  we  can  use  the  method 
of  maximum  likelihood.  We  assume  that  the  ?,t  process  is  normal.  Then,  for 
a  fixed  m,  the  joint  probability  density  function  of  the  variates,  \  +  l , 


'•■+a,  •  •  •>  i=  Given  by: 

+1 .  . . .  ,n  +1  +#/•••  r\  ) 


(/?"*  0.  )■ 


T=T 


30  " 

e  » 


n 

E 

t  =  E  ♦! 


7,ts 


v/hen  the  expected  value  of  :'-t  is  r.i-m  and  its  variance  is  nr  :: ,  Changing 
from  the  7  variables  V.  Hr  X  variables.  nco-.uvling  to  equation  (4.1),  we  have* 


+J  ,  . .  (XB+l  >*8  *S>  •  •  *>xn  I  >:l  'yS  >•  ”>Xr,  ) 

1  „  [  -jS 

|(::t-u)-a1  (xw-u) . aB(xt.B-u))  , 

=  °  ’  (4.  >) 


which  is  the  joint  probability  density  function  of  x>_1,,..,xn  conditional 
on  X_  -Xi ,  X9=xa,  . ..,  Thuc,  to  obtain  the  joint  probability  density 

function  of  Xj ,  Xa,  ...»  X„,  it  is  only  neccr  cry  to  multiply  equation  (4.:  ) 
with  the  density  of  X15  Xa,  . . . ,  X,  .  Since  m  ,  IV  •  ;;  n  ..tiral  applications, 
is  usually  small,  the  not  ''ff.-rt  of  not  carry i.,-  out  this  multiplication  is 
small  and  will  be  omitted.  For  details,  sec  [3('. 


The  Log-likelihood  function  of  the  process  my  be  written  as 

follows : 


L(u, c^jO^,  ...,nT  |  xl  ,xa, . . . ,x*  WU-m^y^n’ -(n-m)0t[  cr, 


z  {(x,  -u)-a!  (xt_j-a)-..  .-a,  (xt_  -u)} 

*  =•  +i  ^  J 


(4.9) 
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The  sum- of -squares  function,  given  by: 

S  (u,04  ,  Rai  •  •  •>(!«  |  ‘  ‘/Xn  ) 

a  .  .8 

f..+i  . a,(*»..-u)}  ,  (4.10) 

is  needed  to  .estimate  the  parameters  of  the  model.  Differentiating 
equation  (4.10),  with  respect  to  n,  <ji ,  cu,  . ..,  a,,  setting  them  equal  to 
zero,  and  solving  the  mtl  system  of  equations,  ve  can  obtain  their  maximum 
likelihood  estimates. 


For  the  second-order  autoregressive  process,  we  differentiate 
equation  (4.10)  with  respect  to  y,  04,  and  &a ,  and  obtain  the  following 
normal  equations: 

a.  (Xg  -p)  »  C4  (X,  -p)  +  (Xj  -p)  , 

b.  £  (x^-P)(xt_x-u)-olI  (Xt.j-p)3  +0feE  (xt_l-p)(xt_aiw)  , 

ui  »*a 


c‘  ^3(x»-^Hxt-a-p)K4^a(xt>1-p)(xt.a-p)+p123(xt.a-p)3  , 


where 


n 

Z  xt-3*j 

n-3  ‘-a 


,  J=l,2,3  • 


Since  Xl,  X,,  and  It,  are  usually  close  to  the  overall  mean,  X,  we  can  use  it 
as  an  approximate  estimate  of  p.  Furthermore,  we  can  obtain  good  approximate 
estimates  of  (a.  through  (c)  using  the  sample  autocorrelation  function  at  lag 
one,  rxs(l).  That  is, 

o,x(l)  *  OlO„(0)  +  p|0„(l)  ,  ~1 

and  .  \  (4.11) 

c„(2)  «  OiCx,(l)  +  aacxx(0)  . 

The  autocovariance  1b  an  even  function,  thus,  we  can  write  equation  (4.11)  as 
follows: 

ck«(J)  M  Oticii(J-l)  +  a^iO-s)  ,  J  ■=  1,2  .  (4.12) 

An  approximate  estimate  of  the  parameters  04  and  is  given  by: 

04  »  (l)Cl.-rx,  (g)^ 

l-^.(l) 
and 

^  «  r*,(2)-i1,(l) 


}  (4.13) 


-471- 


Also,  an  estimate  of  the  residual  sum  of  squares  cam  be  obtained  in  terms  of 
the  sample  autocovariance  function.  That  is: 

S(j!J, <£,<£)  «  (n-2)  {c,,(0)-o^cIl),(l)-a^c!lx(2)}  ,  (k.l*0 

and  the  residual  variance  of  Zt  is  given  by: 

8?  -Hr?  a<tx!  £). 

Similar  expressions  can  be  obtained  for  estimating  the  parameters  for  the 
general  finite  autoregressive  model.  The  normal  equations  may  be  approxi¬ 
mated  by  using  the  sample  autocovarionee  given  by: 

c,„(j)  *  Ohc**(J-l)  +  +  .. .+  (J-m),  (4.15 ) 

J  =  1,  2,  . ..,  m.  Approximate  estimates  can  be  obtained  for  the  parameters 
<tj ,  Og*  . ..,  ot« ,  by  solving  the  m  simultaneous  equations  (4.15), 

The  residual  sum  of  squares  and  the  residual  variance  may  be 
obtained  by  using  the  following  approximations: 

S(ii>ai,  .*.>£  )  W  (n-m)  -...-‘c^cl(l,(m)}  >  (4.15a) 

and 

3?  «  --y-T  s(£  &!>•••>  )  >  (4.15b) 

n-tjrn-i 

respectively. 

4,3  Checking  The  Fit  of  The  Model 

Once  we  have  selected  the  best  process  that  characterizes  the  iono¬ 
spheric  data  and  have  its  parameters  estimated,  diagnostic  checks  are  made 
on  the  model  to  determine  its  adequacy.  Using  this  model,  we  can  obtain  a 
series  that  should  simulate  the  behavior  of  the  original  soundings.  If  the 
original  Goundings  were  filtered,  that  is,  d  was  different  from  zero,  It 
would  now  be  necessary  to  use  a  "backwards  filter, "  replacing  yt  in  the  model 
with  (l-E)dxt,  and  using  the  resulting  process  to  forecast  the  oblique  or 
vertical  incidence  soundings.  For  example,  if  we  fitted  a  first-order  auto¬ 
regressive  model: 

yi-ja  *  dj  (yt_i  -la)  +  zt  ,  ( 4 . 16 ) 

where  yt  *  (1-B)xt  -  xt-xt-1  is  the  filter  UBed  in  the  original  soundings, 
then  inserting  the  filter  into  (4.l6),  we  have; 

=  4b  +  +  %*»-■  +  zt  , 

where  =  £(l-at )  ,  =  (1+a)  and  =  -<a1  .  Thus,  the  mth  order  autoregres 

sive  process,  using  a  first  difference  filter,  can  be  written  as  follows: 

:£t  =  qb  +  tPx xt-i  +  f  •••  +  %*d  +  2^  ,  (4. IT) 

where  the  values  of  (pj  ,  i  <=•  1,  2,  ...,  nu-d,  will  depend  on  m  and  d. 


-47?- 


For  the  fitted  model  to  give ^e.  good  characterization  of  the  iono 
spheric  data,  the  residuals,  rt  =  xt-x*,  t  =  1,  2,  , n,  should  behave 
approximately  like  random  deviates.  Hence,  the  sample  autocorrelation 
function  should  effectively  be  zero  for  all  lags  except  the  zero™  lag. 


4.4  Forecasting  and  Uudat 


the  Model 


One  of  the  aims  in  having  fitted  an  autoregressive  process  to  the 
ionospheric  data,  is  to  forecast  future  values  of  the  oblique  or  vortical 
incidence  critical  frequencies.  If  ve  wish  to  forecast  a  particular  iono¬ 
spheric  sounding,  xt+o, $  £  1,  when  we  are  presently  at  time  slot  t,  then  the 
forecast  is  made  at  origin  t  for  a  lead-time  J{.  Of  course,  the  shorter  the 
lead-time  JJ,  the  more  accurate  our  forecasted  value  will  be. 

The  minimum  mean  square  error  forecast  for  any  lead  time  is  given 
by  the  conditional  expectation,  [1],  Et£Xt  V),  of  xt4.j,  at  time  slot  (origin) 
t,  given  knowledge  of  all  x'a  up  to  time  t.*  That  is, 

Rt  C*t+|]  -  xt(|> 

Replacing  t  with  t  +J^  in  equation  (4.17),  we  have; 

*t+f  =  7b  +  %xt4|  +  +  •••  +  T..d  xt+|-.-d  +  • 

The  minimum  mean  square  error  forocact  of  the  ionospheric  data  is  given  by: 


EtCxt+J{]  . 

7b +74  EtCXt*|-J  + 

. . .+  Erxt 

(4.1(1) 

non-negative  integer,  we  know, 

[1],  that: 

f — 1 

♦ 

* 

X 

1 — » 

fcf 

■  xj(d),  E,[Z,  +  J]  =  < 

);  J  »  lj  2;  •••••  } 

(4.19) 

Rt  Cxt_ j] 

=  xt- j  >  Et CEt- i  1  =  z, 

A 

J  =  Xt-  J  -xt—  J  - 1 

(4.20) 

;,  we  can  write  equation  (4.13) 

as  follows: 

^t  C|)  E  % 

+  cp*  xt*j-i  +  •••  + 

7^  +d  xt+|-#-d  * 

(4.21) 

The  variance  of  the  J|  step  ahead  forecast -error  for  any  time  slot  t;is  the 
expected  '/Blue  of: 


£\(V  -  Cxt+|  -£(i)]n 


(4.22) 


Box  and  Jenkins,  [8],  have  shown  that  the  variance  of  the  lead  time,  H>  iB 
given  by: 


where  Is  estimated  by  sj,  that  is: 

e®  m  S(jjf,>$i  >  •  •  )  f 

1  n 

and  Gj  Is  given  by: 

0,  =0,  j<0 
0Q-1 
«i  -  9i 

ea  -  <ipi  0i  +% 

■ 

• 

6J  -  +  •••  +  9.+d  0j-*-d  •  (b.2k) 

The  (l-a)$  confidence  limits  for  x^+|  is  given  by: 

Pr  {xt(|)-Uj|(l+^“J  0®)^st  i  xt+|Sxtt|  +Ua(i+V  0®  )K}  =1  "a 

2  } 

where  U  is  the  deviate  from  the  unit  normal  probability  distribution. 

ft 

t 


In  ionospheric  problems,  we  are  often  interested  in  forecasting 
future  values  of  an  observed  series  for  several  time  slots  in  advance.  When 
we  forecast  values  at  leady  greater  than  or  equal  t  two  (|fc2)  with  an 
autoregressive  process,  the  forecasted  value  will  be  d  indent  on  previously 
forecasted  values;  but,  as  additional  ionospheric  data  becomes  available,  we 
can  update  our  old  forecast  by: 

xUi  (1)  -  *t(f+l)  +  9jzt+i  • 

That  is,  the  "t"  origin  forecast  of  xt+»+1  can  be  updated  to  become  the 
"t  +  1"  origin  forecast  of  the  Bame  value,  xt+«+1,  by  adding  a  constant 
multiple  of  the  one-step  ahead  forecast  error  Zt+1,  where: 

"  xt+i  -  (f ) 

is  used  with  multiplier  . 
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5.0  AM  AUTORBSRESSIVE  MODEL  FOP.  FORECASTS  OBLIQUE  INCIDENCE  IONOSPHERIC 
SOUNDINGS  OVER  A  6okm'  PATH 


In  this  section,  we  shall  utilize  the  fitting  procedure  of 
Section  4.0  to  develop  an  autoregressive  model  for  forecasting  the  6th  day 
oblique  incidence  (oi)  critical  frequencies  for  the  60Km  experiment.  More 
specifioially,  we  have  available  85  observed  values  of  oblique  incidence 
critical  frequency  data  as  shown  in  Figure  5*1*  We  shall  proceed  by  identi¬ 
fying  the  data  as  a  non- stationary  stochastic  realization,  fit  the  model, 
conduct  a  diagnostic  check  of  the  process,  and  present  both  a  forecasting 
and  updating  scheme. 

5.1  Identifying  The  Observed  Data 

We  plotted  the  data  x,,  t*.l,  2,  ...,  85  to  attempt  to  visually 
detect  any  trend  or  non- randomness  (sec  Figure  5.1).  The  graph  of  the  data 
appears  to  exhibit  non- stationary  properties.  As  a  further  aid  in  identi¬ 
fying  the  data,  we  calculated  the  sample  autocorrelation  function  of  xt  and 
conducted  statistical  tests  for  trend.  The  sample  autocorrelation  function, 
c  (k),  was  calculated  for  the  original  data  and  for  the  first  and  second 
difference  filters.  These  calculations  are  shown  in  Tablen  5*2,  5*3/  and  5>4, 
respectively. 

It  is  clear  that  the  sample  autocorrelation  function  of  the  second- 
difference  data  dampens  out  fairly  rapidly,  ind  ha  ting  that  the  filtered 
series,  wt ,  have  reached  statistical  equilibrium,  furthermore,  wc  performed 
statistical  tests  using  Kendall's  tau  test  and  found  that  at  the  5 1>  level  of 
significance,  the  original  data  contained  trend.  Tliat  in,  for  the  first 
difference  data,  yt ,  the  tost  statistic,  war.  found  to  be  - 6 . 5«?9 J  and  for 
the  second  difference  filter,  it  was  -0,578,  which  indieaten  trend  at  the  % 
level  of  significance  (a<0B  =  *1.645), 

Therefore,  the  day  01  critical  frequencies  recorded  for  the 
60Km  experiment  constitute  a  non- stationary  time  series.  A  second  difference 
filter  will  transform  the  data  into  statistical  equilibrium. 


5 .2  Fitting  the  Autoregressive  Model 


Using  the  filtered  series  wt ,  t«l,  2,  85,  we  shall  fit  an 

autoregressive  process.  Recall  that  in  order  to  fit  such  a  model,  it  is 
necessary  to  estimate  the  parameters  for  processes  of  different  ordere,  and 
then  compute  the  residual  variances  for  each  order.  With  this  information, 
one  can  decide  on  the  order  of  the  autoregressive  process  which  best  fits 
the  recorded  data.  Using  the  maximum  likelihood  equations  approximation, 
(4.15),  we  calculated  estimates  of  the  parameters,  the  residual  sum  of  square^ 
(4.15a),  and  the  residual  variance,  (4,15b),  of  the  soundings  for  the  auto¬ 
regressive  processes  of  orders  1,  2,  ...,  10.  Figure  5.5  shows  the  residual 
variance  plotted  against  the  order  and  also  chows  that  the  minimum  residual 
variance  corresponds  to  an  autoregressive  process  of  order  two.  Therefore, 
a  second  order  autoregressive  process  will  give  the  best  fit  to  the  filtered 
6tl  day  01  data.  Using  the  procedure  discussed  in  Section  4.0,  the  estimates 
of  the  true  state-of -nature  of  the  parameters  of  such  a  model  xre re  found  to 


be; 
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9 


a  B  3 

\X  =  |l  ^  -  0.004 
sl  es 

n.x  c  0.409  2 


and 


£  -  0.255  . 


Thus,  the  autoregressive  model  for  the  filtered  6th  day  data  is; 

wt  -  .004  b  -.409  (vt_i-*004)  -  .255(wt_a-.004)  +  zt  (5.1) 

Note  that  the  parameter  estimates  satisfy  the  condition  that  they  must  lie 
within  a  unit  circle. 

5*3  Diagnostic  Check  of  the  Autoregressive  Model 
Inserting  the  backward  filter: 
xt  -2xt-i+xt-A  *  wt 

into  the  filtered  model  (5.1),  we  have: 

^-2xt-i+xt_ft-.004  m  -.409(xt„l -2xt_a+xt-3 -.004) 

-•255(x4.a-2xt.3+x4_4-.004)+  zt  (5.2) 

Simplifying  equation  (5 .2)  we  obtain  the  ''^recasting  model  for  the  6th  day 
01  data: 


xj  =  .006  +  I.591  x4-1-.43  xt_g+. 1x4.3 -.255  xt-*+zt  (5.3) 

For  simplicity,  we  write  equation  (5*3)  as  follows: 

*t  =  db+  T>ixt-i+«fii3S-a+«text-3+<^xt-4+zt  (5» 

where : 

%  «  t^i-Oi-a^)  =  ,006 

tfii  “  2  +  *  1*591  •» 

)  (5*5) 

m  a,, -2cq -1  «  -.^30 

efts  -  ®i  *2aj|  *  .1 


CCk  *  “  "  *255 

To  simulate  x^ ,  we  use  (5*5)  set  the  unknown  value  z%  equal  to  its  uncondi¬ 
tional  expectation  of  zero,  and  assume  the  values  xt_x,  xt_g,  xt_3,  and 
are  known.  Figure  5*6  shows  the  simulation  of  the  observed  series, 
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t«l,  2,  ...,05.  We  obtain  the  residuals  by  subtracting  the  modeled 
series  from  the  observed  series,  that  is: 

rt  ■  Xt  -% ,  t«5,  . ..,  B5  (5.6) 

The  residuals  start  at  time  slot  5,  and  rx  ■  -  ra  *  rA  =  0.  That  is,  it 

is  necessary  that  the  first  four  values  of  the  data  be  known  and  that  fore¬ 
casting  begins  at  the  next  time  slot.  The  residual  sum  of  squares  was  found 
to  be: 


IT  r=  -  1.991*  (5.7) 

<•!) 

The  behavior  of  the  residuals  approximates  that  of  a  purely  random  process, 
where  their  sample  autocorrelation  function  should  be  effectively  zero. 

Figure  5.7  Bhows  that  we  have  developed  a  good  model  for  the  ionospheric 
soundings  over  a  60KW  path. 

Since  n  is  sufficiently  large,  the  sample  autocorrelation  function, 
r  (k)  is  approximately  normal  with  mean  zero  and  variance  l/n,  £)' J.  The 
standard  deviation  of  r,,(k)  was  found  to  be  0.11  and  the  95$  confidence 
limits  for  r,,(k)  of  the  residuals  is: 

r, , (k)  ±  p.ll  (I.96)  1 
or: 

PrCr«i  (k)-.82  *  p, , (k)  £  r„(k)  +  .22]  *  .95, 
or  see  Figure  5.7. 

5.4  Forecasting  and  Updating 
The  fitted  model: 

xj  -  cfc  +  CRiXt_1+(^xt_lt+%xt_3+cptxt_4+zt 

may  be  used  to  forecast  future  values  of  the  observed  series  xj .  To  forecast 
ahead  for  a  lead  time,  J|,  we  have; 

The  minimum  mean  Bquare  error  forecast  is  given  by: 

S,<|)  ■  Cfe+tPlXt+|-l+VtXt+J-Ji+%xt+|-3+5fcXt+|-4  * 

To  illustrate  how  one  may  update  the  forecasts  for  a  slot  time  t, (origin) 
Buppose  that  a  new  piece  of  data  ,  xt+l ,  becomes  available.  With  the  origin 
at  time  slot  t+1,  we  update  the  forecast  by: 

^  (j^+l)  4  6jj  zt  l  m  1,  2,  ...,  10 

where:  z,+1  *  xt+J  *1^(1)  and  tk  is  as  described  in  Section  4,0. 


471- 


Table  5.8  shows  the  forecasted  values  of  the  6th  day  oblique 
incidence  series  for  1,  2,  11  tine  slots  ahead  at  origin  t*59  along  with 

the  50$  and  95$  probability  limits.  The  actual  values  of  x*  are  shown  for 
comparison,  but  these  values  are  not  actually  known  when  the  forecast  la 
made.  It  also  shows  the  updated  forecast  for  1,  2,  ... ,  10  time  slots  ahead 
at  origin  t-60.  Note  that  the  updated  forecast  is  an  improvement  over  the 
original  forecast  at  every  lead  time. 
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FIG,  5.5  RESIDUAL  VARIANCE  OF  THE  AUTOREGRESSIVE 
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FIG.5.T  SAMPLE  AUTOCORRELATION  FUNCTIONS  AND  95%  CONFIDENCE 

INTERVAL  OF  THE  RESIDUALS  OF  THE  OBLIQUE  INCIDENCE  CRITICAL  * 
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FORT  MONMOUTH,  N.J.  -  FORT  DIX ,  N.J 
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In  this  section,  ve  shall  illustrate  the  fitting  procedure 
discussed  in  Section  4  by  formulating  a  difference  equation  for  the  mean 
vertical  incidence  soundings  for  the  60  Km  experiment.  More  specifically, 
ve  shall  fit  a  moving  average,  model  to  the  85  observed  values  of  the  vertical 
incidence  data,  each  of  which  is  the  mean  of  nine  observed  soundings  taken  at 
specific  time  slots  during  the  period  of  the  experiment. 

6.1  Identifying  The  Series; 

A  plot  of  the  data,  x%l  t»l,  2,  ...,  85,  is  shown  in  Figure  6.1. 

The  visual  Interpretation  of  the  time  series  1b  that  it  exhibits  non- 
itationary  properties.  To  substantiate  this,  ve  calculated  the  sample  auto¬ 
correlation  function.  Table  6.2  shows  that  the  aaniple  autocorrelation  does 
not  dampen  out  very  rapidly.  Furthermore,  Kendall's  tau  test,  [3,  for  trend, 

was  applied  and,  at  the  3%  level  of  significance,  we  confirmed  the  fact  that 

the  ionospheric  data  was  not  in  statistical  equilibrium. 

A  first  difference  filter: 

yt  ■  x1  -  1  t  »  1,  2,  ...,  85, 

and  a  second  difference  filter; 

vt  *  —  2xt_1  +  xi ^ ^ j  t*  1,  2,  ..*,  85, 

were  applied  to  the  original  data.  Using  the  second  order  difference  filter, 

the  sample  autocorrelation  function,  shown  in  Table  6.4,  dampens  out  more 
rapidly  than  if  the  first  difference  filter  (shown  by  Table  6.3)  were  used. 
Also,  the  values  of  Kendall's  tau  are  -6.274  and  -0.539  for  the  first  and 
Becond  difference  filters,  respectively.  Thus,  at  the  5$  level  of  signifi¬ 
cance  (z.0B  ■  *  1.645),  the  second  difference  data  indicates  that  the  VT 
soundings  are  in  statistical  equilibrium. 

6.2  Fitting  the  Moving  Average  Model: 

Using  the  maximum  likelihood  method,  we  estimated  the  parameters 
of  the  moving  average  model  for  orders  up  to  five,  and  then  we  computed  the 
residual  variance  for  each  order.  As  indicated  above,  the  criterion  for 
selecting  the  order  of  the  process  which  gives  the  best  fit  to  the  mean 
vertical  incidence  data  will  be  the  order  which  has  minimum  residual 
variance. 


Figure  6.5  shews  that  the  minimum  residual  variance  for  the  moving 
average  process  will  be  given  by  order  two.  Hence,  a  second-order  moving 
average  process  will  give  the  best  fit  to  the  filtered  series,  wt .  The 
parameters  estimated  for  this  model  were  found  to  be: 

ti  «  0.0  ,  k  O.58  ,  and  ^  *  -0.12 
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Note  that  the  invortibility  conditions  discussed  above  are  satisfied  by  the 
estimates  of  the  parameters.  Thus,  the  moving  average  process  that  charac¬ 
terizes  the  filtered  data  is: 

_ 

V|  *  ^  •  12  a 

6.3  Diagnostic  Check  Of  The  Model: 

To  determine  the  adequacy  of  the  fitted  process,  ve  must  simulate 
the  observed  series,  and  then  calculate  the  residuals  to  see  if  they  behave 
as  a  purely  random  process.  Recall  that,  in  order  to  use  the  above  model, 

,  to  simulate  the  observed  series,  x, ,  ve  must  make  use  of  the  "backwards 
filter"  which  depends  on  the  original  filter  employed  to  transform  the 
observed  soundings.  Hence,  inserting  the  backward  filter: 

~  2xt-i  +  “  wt 

to  our  model,  w^,  we  have: 

xj  *  2xt-i  "  xt-a  +  zt  "  ’38  2i-i  +  *12  zt_a  . 

Setting  the  unknown  values  of  zt  equal  to  their  unconditional  expectation  of 
zero,  we  begin  the  simulation  by  initially  assuming  xx  and  x_  are  known;  to 
simulate  xt ,  we  assume  x,_1  and  xt_fc  are  known.  Figure  6.6  shows  the 
simulated  series  along  with  the  original  data. 


To  check  if  the  residuals  behave  as  a  purely  random  process,  we 
calculated  the  sample  autocorrelation  function,  (k),  of  the  residualsifor 
lags  k  ■  1,  2,  ...,  84.,  Figure  6.7  shovB  that  r^rk)  indeed  dampens  out 
rapidly. 


Since  the  total  number  of  ionospheric  soundings  is  sufficiently 
large,  n  -  85,  r, , (k)  is  approximately  Gaussian  with  mean  zero  and  variance 
l/n.  The  calculated  value  of  the  standard  deviation  of  rt, (k)  was  found  to 
be  0,11  and  the  9 %  confidence  limits  are: 

rtl(k)  ±  .22  . 

Figure  6.7  shows  that  none  of  the  Bample  autocorrelations  of  the  residuals 
are  outside  the  above  confidence  limits. 

6.4  Forecasting  And  Updating: 

To  forecast  ahead  "JJ"  slots  with  the  above  moving  average  model, 

we  have : 

=  2jci+|-1-xi+|_#*zt+|-.58zt4|_1+  .12zt+|_j  . 

The  minimum  mean  square  error  forecast  is  given  by: 

*»  (9)  E  2xt  +jf-i  -xt  +J-R,"  .5®zt  +|-i+  .l^Zt  +  • 
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Using  $,(j()  to  forecast  lead  times  greater  than  two,  the  model  becomes: 

*(f)  •  2*»+|-x -*»♦/-*  »  i>2  • 

Mote  that  the  forecasted  values  will  not  depend  on  previous  errors,  zt . 

To  update  the  forecast  made  at  origin  t,  we  assume  that  a  new 
ionospheric  sounding,  2^.,  has  been  realized.  Now,  with  the  origin  at  t+1, 
itc  uay  update  the  forecast  using: 

($)  -  d+1)  ♦  9|C*t+i-xJ  (1)3  > 

where  the  0|  are  as  given  in  Section  4, 

Table  6.8  shows  the  forecasted  values  of  the  data  for  1,  2,  ...,  5 
tine  slots  ahead  at  origin,  t  -  1*0,  along  with  the  associated  50 %  and  95# 
confidence  limits.  The  actual  values  of  the  observed  series  are  shown  for 
comparison,  but  they  were  not  actually  known  when  the  forecast  was  made.  The 
table  also  shows  the  updated  forecasts  for  1,  2,  3,  end  k  time  Blots  in  the 
future  at  origin  t  ■  4l. 
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FIG,6.J>  RESIDUAL  VARIANCE  OF  THE  MOVING 
AVERAGE  MODEL,  ORDERS  1-5,  OF  THE 
OVERALL  MF4N  OF  VERTICAL  INCIDENCE 
CRITICAL  FREQUENCIES  TAKEN  AT 
FORT  MONMOUTH  ,  N.J.  FOR  THE 
60  Km  EXPERIMENT. 
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OBSERVED  SERIES  OF  THE  OVERALL  MEAN  OF  THE  VERTICAL 
INCIOENCE  CRITICAL  FREQUENCIES  TAKEN  AT  FORT  MONMOUTH,  N.4. 
FOR  THE  60  K*  EXPERIMENT 
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7.  SUMMURY  AMD  C0NCUU8I0K3 


Oblique  Incidence  (OZ)  and  vertical  Incidence  (VI)  Ionospheric  sound Inga 
were  obtained  aa  described  in  section  2  over  60  job,  200  KB,  and  $00  Km 
distances,  me  min  objectives  of  these  experiments  vers  to  develop  a  fore¬ 
casting  model  to  predict  short-term  future  ionospheric  conditions  for  these 
specific  distances,  knowing  VI  data  at  a  particular  time,  and  to  estimate 
equivalent  OZ  data  over  the  area  of  interest.  In  view  of  these  experimental 
investigations,  over  the  paths  specified,  the  following  results  were  obtained 
along  with  the  related  conclusions; 

(a)  It  was  pointed  out  tAst  the  widely  accepted  approach  (secant 
law)  of  estimating  equivalent  01  froa  VI  data  for  whet  are  considered  tactical 
Field  Army  distances  (especially  at  500  Km),  does  not  give  the  desired 
accuracy  for  practical  use  within  the  Field  Army  area  of  responsibility. 

(b)  A  table  has  been  formulated  (p.  8  )  which  shows  that  at  the 
three  distances  investigated,  there  exists  a  very  st rarq  linear  dependence 
between  observed  01  and  VI  data. 

(c)  In  view  of  the  strong  linear  dependence  demonstrated,  linear 
regression  models  were  developed  for  all  three  distances.  More  specifically, 
two  regression  models  were  developed  for  each  of  the  distances  investigated 
(and  also  for  each  of  the  reciprocal  paths),  namely,  one  for  a  specific  day, 
chosen  at  random,  and  one  for  the  overall  mean  of  the  observed  data. 

(d)  It  has  been  shown  Id  '->?ction  3,  that  one  can  obtain  an  excel¬ 
lent  estimate  of  the  equivalent  Of  data  at  a  particular  time  slot,  for  a 
given  path,  knowing  the  VI  information. 

(e)  For  each  of  the  linear  regression  models  developed,  we  gave 
95/6  confidence  Intervals.  That  is,  we  are  95$  certain  that  these  bounds 
contain  the  true  state  of  nature  which  ve  have  estimated. 

(f )  One  could  use  the  graphical  presentation  of  the  regression 
models  (pp.  10-21  )  to  estimate  equivalent  01  data.  For  the  experiments 
performed,  these  results  can  be  directly  compared  to  the  actual  observed  01 
data, 

(g)  One  of  the  needs  of  the  tactical  Field  Army  for  more  effective 
H.  F.  connunleatlonB,  is  to  develop  a  system  to  forecast  the  VI  soundings  at 
specific  times  In  advance  for  specific  distances.  Haring  such  a  forecasted 
value  at  a  particular  time,  one  can  insert  it  into  a  regression  model  to 
obtain  an  estimate  of  the  equivalent  01  soundings  at  the  same  time  in 
advance. 

(h)  In  view  of  the  above  need,  a  precise  procedural  approach 
(Sections  5  and  6),  utilizing  time  series  analysis  techniques,  has  been 
given  for  the  development  of  such  forecasting  models. 

(i)  A  complete  analysis  using  the  above  procedure  was  carried  out 
for  the  60  Kta  experiment.  It  was  shown  that  both  the  01  and  VI  soundings 
vere  non- stationary  stochastic  realizations  and  filtering  of  the  data  was 
necessary. 
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(J )  An  autoregressive  model  was  developed  for  forecasting  01 
soundings  over  a  60  Km  path.  This  resulted  In  the  following  difference 
equation: 

.006  +  1.591  p  +  .10  x^.3~.255  • 

One  can  utilise  this  model  to  forecast  the  01  recording  at  1,  2,  ...,  k  tine 
slots  In  advance.  It  is  necessary  that  ve  have  four  (4)  Initial  values  of 
the  experiment . 

(k)  A  method  has  also  been  given  where  the  model  can  be  updated. 
That  is.  If  additional  information  becomes  available,  it  can  be  utilized  to 
improve  the  forecasted  values  for  many  time  slots  in  advance. 

(l)  A  table  has  been  formulated  (p.  4l)  where  ve  utilised  the 
autoregressive  model  to  forecast  01  data  up  to  ten  time  slots  in  advance. 

Also  given  are  5 <#  and  95 i  confidence  bounds  of  the  true  state  of  nature 
associated  with  this  physical  phenomenon. 

(m)  A  moving  average  model  has  been  developed  for  forecasting 
vertical  incidence  (VI)  ionospheric  soundings  for  the  60  Km  experiment  at 
the  Port  Monmouth  terminal. 

(n)  As  was  the  case  with  the  01  data,  the  VI  data  exhibited  non* 
stationary  properties.  A  second  order  filter  was  necessary  to  transform  the 
data  into  statistical  equilibrium 

(o)  The  moving  average  model  which  characterizes  the  VI  soundings 
is  given  by:  <*. 

X|,  *  ^  . 

One  can  utilize  the  above  model  to  obtain  a  VI  value  at  a  given  time  slot  in 
the  future,  and  by  using  this  value  in  the  corresponding  linear  regression 
model,  we  can  obtain  the  necessary  future  estimate  of  the  equivalent  01 
sounding. 

(p)  The  sample  autocorrelation  function  of  the  residuals,  that  is, 
the  actual  value  minus  the  forecasted  value,  was  calculated.  It  dampened 
out  fairly  rapidly,  which  indicates  the  effectiveness,  in  terns  of  accuracy, 
of  the  proposed  model. 

(q)  A  table  has  been  given  (p.  52)  where  the  aovli«  average  model 
has  been  utilized  to  forecast  VI  data,  1,  2,  ...,  6  time  slots  in  advance. 

Also  given  are  the  50%  and  95%  confidence  intervals. 

The  findings  and  the  models  that  have  been  developed  in  this  study  can 
be  utilized  by  the  tactical  Field  Any  for  the  approximate  distance  Involved 
under  similar  circumstances  of  geographical  location,  seasonal  effects,  and 
geomagnetic  activity.  This  is  not  to  say  that  the  models  could  not  be  revised 
so  that  they  could  be  universally  used  for  this  tactical  distance. 

The  authors  are  presently  involved  in  formulating  statistical  forecasting 
models  for  the  200  Km  and  500  Km  experiments. 
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Abstract 


Ultimate  development  of  military  veapona  flaah  detection/location  equip¬ 
ment  ia  hindered  by  a  lack  of  engineering  data  characterialng  the  AC 
component  of  the  infrared  background. 

Experimental  determination  of  the  needed  data  haa  proved  very  difficult 
due  to  problema  in  conceiving  of  an  experiment  which  can  aeparately 
measure  all  the  AC  background  components  (a,g.  background  changes  are 
indistinguishable  from  atmospheric  turbulence,  etc).  Without  such 
separation  L-f  effects,  it  is  impossible  to  reduce  results  to  terms 
of  engineering  interest. 

Requirements  on  the  needed  data  and  experimental  equipment,  and  pro¬ 
cedures  are  discussed. 


IHTRODUCTION 

Military  requirements  exist  for  systems  to  detect  and  locate  nbqtile 
ordinance  of  various  types.  In  recent  years  several  possible . solutions, 
to  the  problem  have  been  investigated.  The  detection  and  location  of 
weapons  by  their  muazle  flash  has  proved  to  be  the  most  feasible  approach. 
Recent  studies  have  shown  that  staring  systems,  operating  in  the  near  in¬ 
frared,  are  superior  to  other  proposed  approaches  to  flash  detection/loca¬ 
tion. 


The  detection  capabilities  of  such  systems  are  limited  under  daylight 
conditions  by  the  AC  component  or  transient  nature  of  the  background  radi¬ 
ation.  Several  difficulties  have  been  encountered  in  our  initial  attempts 
to  characterise  this  phenomenon. 

These  difficulties  seem  to  Involve  two  distinct  problem  areas  for 
which  outside  aid  and  guidance  is  sought.  First,  we  lack  a  theoretical 
explanation  of  the  phenomenon  under  investigation.  Without  such  a 
theory  it  is  difficult  to  design  definitive  measurement  experiments. 
Second,  we  need  advice  on  aviilable  data  reduction  procedures  to  insure 
the  generality  and  utility  of  our  data. 


Inscription  of  Staring  Flash  Detection  System 

A  aimpla  flash  detection  system  consists  of  optica,  s  photodeteotor, 
and  associated  electronics. 

Tbs  optics  colleot  light  and  focus  it  on  the  photodeteotor.  The 
ratio  of  collecting  area  to  deteotor  area  detemlnes  the  optiaal  gain. 

The  physical  dimensions  of  the  detector  and  the  focal  length  of  the  op¬ 
tics  fixes  the  field  of  view  of  the  system.  Targets  outside  this  field 
of  Ties  are  not  imaged  on  the  deteotorj  hence  are  net  detected.  An 
optioal  filter  Is  used  to  pass  light  within  a  specified  wavelength 
region  (the  optical  passband)  and  to  attenuate  all  other  wavelengths. 

The  photodeteotor  provides  an  eleotrical  output  which  is  propor¬ 
tional  to  the  flux  density  of  light  (irradianoe)  Impinging  on  it. 

Threshold  detection  eleotronics  for  such  systems  may  be  quite  com¬ 
plex.  The  simplest  version  would  Include  an  electronic  filter  olrcult 
and  a  threshold  deteotor.  The  electronic  filter  is  used  to  optimise 
the  signal  to  noise  ratio.  Spurious  nolss  signals  gsnsrstsd  hy  the 
baakgrpund  or  the  photodeteotor  are  to  be  attanuated  as  much  as  possible. 
Many  wall  known  criteria  are  era  liable  to  gulds  filter  design  if  tbs 
statistical  natures  of  tha  signal  and  noisa  are  known. 

The  threshold  deteotor  passes  only  those  signals  which  Satisfy  a 
given  deteotion  oriteria.  Tha  simplest  of  those  conditions  is  that 
the  output,  of  the  deteotor  exoaede  a  threshold  level.  However,  deteo¬ 
tion  oriteria  cun  also  be  v»iy  complex.  It  is  possible  to  design 
discrimination  electronics,  if  ths  statistical  natures  of  ths  signal 
and  noise  are  known. 

It  should  be  noted  here  that  since  we  are  only  interested  in 
detecting  transients,  ths  photodeteotor  is  AO  coupled  to  the  eleotronios 
to  allow  deteotion  against  any  constant  intensity  background. 


Requirements  for  System  Modeling  and  Psalm 

Ths  probability  dsnsity  function  of  ths  nolss,  at  ths  input  to 
tha  threshold  detector,  detemlnes  the  threshold  settings  required 
to  produce  a  given  level  of  performance.  Similarly,  the  power  spec¬ 
trum  and  time  correlation  function  of  tha  noise,  at  the  input  of 
the  electronic  filter,  determines  the  passband  of  maximum  signal  to 
nolss  for  a  given  signal. 
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The  noise  input  to  the  filter  is  e  sun  of  two  components.  One 
component,  the  dark  noise  of  the  deteotor  end  electronics,  depends 
entirely  on  the  type  of  detector  and  oirouits  used.  The  characteris¬ 
tics  of  this  component  may  be  estimated  from  knowledge  of  the  deteotor 
material  and  configuration  and  a  circuit  diagram.  They  may  also  be 
measured  for  any  completed  device.  If  compensation  for  varying  de- 
tector/olrcult  temperature  ie  allowed,  theee  characteristics  are  con¬ 
stants  of  the  deviee. 

The  other  component  of  the  noiee  ie  oaused  by  the  random  or 
pseudorandom  fluctuations  in  the  irr ad lance  etriklng  the  optical  fil¬ 
ter.  The  portion  of  these  varying  light  levels  within  the  optloal  pass- 
band  of  the  filter  ie  paseed  on  to  the  optica  which  images  it  on  the  de¬ 
tector.  The  deteotor  converts  it  to  variations  in  its  output  level. 

These  unwanted  fluctuations  are  passed  on  to  the  detection  oirouits. 

At  this  point,  they  combine  with  the  dark  noise  to  mask  the  weapons 
signals  wo  wish  to  detect  and  produce  false  alarms.  It  is  these  fluc¬ 
tuations  in  light  intensity  that  we  must  characterise  and  include  if 
we  are  going  to  predict  system  performance. 

There  are  two  broad  categories  of  phenomena  which  cause  the  fluc¬ 
tuations  in  light  intensity.  Variations  in  the  amount  of  light  reflected 
from  the  terrain  cause  ohanges  in  the  background.  Examples  include 
sunglints  from  water,  shadows  of  passing  clouds,  and  waves  on  tall 
grass  in  the  wind.  Ohanges  in  the  atmosphere  also  produce  variations 
in  the  amount  of  light  reaching  the  optics. 

It  is  Important  to  keep  in  mind  the  dif ferenoes  in  these  two  phe¬ 
nomena.  The  total  change  in  the  terrain  itself  can  be  \assumed  to  be 
the  total  of  all  the  changes  in  all  the  individual  components  of  the 
terrain.  Variable  refi motions  off  terrain  have  definite  magnitudes 
and  spatial  characteristics  at  their  souroes.  For  instance,  a  sun 
glint  off  water  has  a  definite  optical  speotrum,  temporal  signature, 
Intensity,  and  a  limited  spatial  extent  at  the  wave  which  creatos  it. 
These  variations  will  decrease  in  magnitude  and  may  ohange  in  other 
ways  with  the  distance  to  the  particular  background  disturbances  causing 
them. 


Fluctuations  caused  by  turbulence  in  the  atmosphere  are  not  so  easy 
to  deal  with.  For  instar.oe,  it  is  not  at  all  clear  how  the  Intensity  of 
such  fluctuations  ohange  with  distance  to  the  limit  of  the  field  of  view. 
Furthermore,  since  the  light  from  the  terrain  must  pass  through  the 
turbulent  atmosphere,  it  is  clear  that  the  two  types  of  noise  are  con¬ 
volved  not  added. 

The  problem,  then,  is  to  measure  the  fluctuations  of  light  induced 
by  variable  reflections  off  the  terrain  and  a  turbulent  atmosphere. 

The  statistical  characteristics  of  baokground  noise  must  be  known  for 
any  set  of  system  and  background  conditions. 


-503- 


W#  must  be  able  to  eetlnate  such  thing*  as  the  probability  density 
function  of  the  background  noia*.  Its  power  spectrum  and  time  correlation 
function  also  must  be  known.  Xn  addition  we  must  know  how  these  charac¬ 
teristic  s  change  with  various  optical  paasbands  and  fields  of  view. 

Some  detection  systems  oonoepts  are  susceptible  to  variation  In  the 
shape  of  the  image  of  the  disturbance  on  the  detector.  Therefore,  we 
must  know  the  shape  and  sis*  of  ths  souroe  of  the  light  fluctuation. 

This  information  would  be  beneficial  in  distinguishing  between  terrain 
and  atmosphere  induced  variations.  We  would  also  like  to  determine 
whether  any  olassee  of  background  transients  exist  which  exhibit  similar 
temporal  or  spectrum  signatures.  Xf  such  classes  do  exist  it  would  be 
Important  to  determine  the  statistical  nature  of  their  signatures  so 
that  possible  discrimination  schemes  could  be  evolved. 


Petting  ths  Data 
What  Has  Been  flone 

To  date  very  little  work  has  been  don*  in  this  area.  The  most 
closely  related  field  of  aotlv*  endeavor  has  been  the  effects  of  atmos¬ 
pheric  turbulence  on  beams  of  light.  Most  of  this  work  has  been  direoted 
toward  investigation  and  explanation  of  beam  wander  and  non-uniform  light 
Intensities  associated  with  the  transmission  ot  such  beams  through  the 
atmosphere . 

Perhaps  the  single  moat  comprehensive  work  In  this  area  in  that  of 
7.  I.  Tatarski  of  the  Institute  of  Atmospherlo  Physios,  Aoademy  of  Soienoes 
of  the  USSR.  In  his  book  "Wave  Propagation  in  a  Turbulent  Medium'1, 

Tatarski  attempts  a  thorough  theoretical  explanation  of  the  phenomenon 
of  atmospheric  solntlllatlon.  This  is  the  phenomenon  which  causes  the 
twinkling  of  the  stars.  He  also  attempts  to  establish  agreement  between 
his  theoretical  and  experimental  results. 

Several  others  have  made  photographic  studies  of  the  shapes  of  sha¬ 
dow  patterns  produoed  by  atmospheric  solntlllatlon  on  a  beam  of  light. 

These  studies  show  amorphous  dark  patterns  which  move  across  ths  beam 
with  a  velocity  approximately  equal  to  ths  component  of  wind  velocity 
in  that  direction. 

It  la  Important  to  note  here  that  such  studies  are  of  limited  appli¬ 
cation  to  staring  systems.  Scintillation  experiments  measure  tho  tem¬ 
poral  and  spatial  variations  imposed  on  an  unmodulated  beam  due  to  trans¬ 
mission  through  the  turbulent  atmosphere.  As  such  they  are  only  appli¬ 
cable  to  a  single  point  source  of  radiation  at  a  known  distance  from  the 
detector.  Furthermore,  the  nature  of  these  experiments  is  such  that 
light  entering  the  detector  from  points  other  than  the  experimental  source 
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is  rejected  or  Ignored.  Thus  the  oonospt  of  f laid  of  view  doss  not  apply. 

A  staring  dsteotion  system  nay  be  considered  as  a  mosaic  detector 
located  In  the  focal  plane  of  oolleotlng  optics.  Baoh  elemental  detector 
has  a  specific  field  of  view.  The  total  system  field  of  rise  is  the  sum 
of  the  elemental  fields  of  view.  It  is  conceivable  that  both  spatial 
and  angular  correlation  functions  are  involved  in  characterising  back¬ 
ground  fluctuations. 

Finally  there  is  a  very  great  difficulty  in  analysing  radiometric 
data  of  this  nature.  For  although  it  is  olearly  possible  to  measure 
spatial,  angular,  and  time  correlation  functions,  probability  density 
functions,  and  potter  speotral  of  signals  from  a  mosaic  of  detector  ele¬ 
ments,  there  is  no  obvious  Hay  of  separating  the  effects  of  terrain  and 
those  of  the  atmosphere  once  the  data  is  taken.  In  fact,  we  do  not 
now  understand  how  two  convolved  effeots  can  be  "unconvolved"  or 
separated. 

This  difficulty  is  also  increased  by  the  fact  that  the  noise  of  the 
measuring  system  la  inoluded  in  such  test  results.  Normally,  to  avoid 
such  difficulties  one  merely  makes  the  measuring  equipment  many  times 
mors  sensitive  than  tha  detection  equipment.  However,  in  this  case,  the 
detection  equipment  is  already  pushing  the  theoretical  limits  of  sensi¬ 
tivity  for  equipment  of  this  type.  Thus,  the  best  measurements  whioh 
can  be  made  must  either  be  made  under  extremely  noisy  background  condi¬ 
tions  or  with  a  low  background  signal  to  system  noise  ratio. 


Our  Work  to  Date 

A  large  part  of  our  time  and  effort  is  spent  measuring  gun  flashes. 
These  measurements  are  made  with  radiometers  which  are  quite  similar  to 
staring  dsteotion  systems.  When  we  noticed  very  noisy  background  con¬ 
ditions  we  recorded  the  resultant  signals  for  30  seoonds  or  so.  We 
then  recorded  an  approximately  equal  length  of  data  from  the  radio¬ 
meters  with  the  Input  aperatures  completely  blocked. 

Oir  intention  waa  to  consider  the  differences  in  the  power  spectra 
of  tha  two  sets  of  data.  The  difference,  it  is  hoped,  will  yield  the 
power  apeotrum  of  the  background  (terrain  plus  atmosphere)  component 
lees  measurement  system  noise.  Thus,  stationarity  of  the  measurement 
system  noise  must  be  assumed.  Presuming  both  background  and  system 
noise  to  be  ergodio  makes  any  measurement  of  the  probability  density 
function  much  simpler.  However,  it  is  not  clear  at  this  time  what 
rational  basis  might  be  used  to  separate  the  effects  of  system  noise 
from  the  probability  distribution.  This  is  complicated  by  the  fact 


-505- 


that  tha  AC  coupling  tends  to  bring  tha  maana  of  all  suoh  functions 
to  sero. 

Sines  ks  used  single  dataotion  radioes tars  each  with  a  diffsrsnt 
optical  paaaband  no  spatial  or  angular  data  ia  available.  Nor  la  any 
attempt  being  made  to  separata  atmoepherle  affects  from  those  of  tha 
terrain. 

We  are  currently  endeavoring  to  design  equipment  more  nearly 
suited  to  these  type  measurements.  This  equipment  will  be  based  on 
an  unusual  deteotor  developed  for  us  by  Minneapolis  Honeywell.  The 
deteotor  is  made  of  two  layers  of  three  bars  eaoh.  The  bare  of  tha 
upper  layer  are  at  right  angles  to  those  of  the  lower  layer.  The 
layers  are  held  together  by  a  transparent  insulating  epoxy. 

The  two  detector  layers  are  made  in  suoh  a  way  that  the  top  layer 
responds  to  and  absorbs  one  set  of  optical  wavelengths  while  it  is 
transparent  to  the  wavelengths  to  which  the  bottom  layer  responds. 

With  this  deteotor  it  should  be  possible  to  measure  the  angular 
correlation  function  over  a  3  eleoant  spacing.  This  could  be  done  in 
two  optlaal  passbanda  simultaneously  and  in  the  asm  field  of  view 
and  from  the  asms  location.  With  suoh  information  we  hope  to  begin 
trying  to  separate  the  effeots  of  terrain  from  those  of  atmosphere. 

It  seems  knowledge  of  the  angular  else  of  the  disturbance  may  allow 
us  to  separate  those  effeots  produced  by  wide  shifts  of  meteorological 
conditions  across  tha  field  of  view.  If  detector  arrays  with  more  ele¬ 
ments  become  available  it  should  be  possible  to  use  visual  sightings 
of  events  within  the  field  of  view  to  aid  this  separation  effort.  How¬ 
ever,  since  tha  two  components  are  convolved  within  any  elemental 
field  of  view  we  know  of  no  way  to  complete  this  prooeas. 

Summary  of  Heeds  Thus  a  summary  of  our  nasds  to  complete  the  design  of  a 
successful  experiment  includes > 

1.  Some  means  of  "unconvolvins"  two  effects  when  some  characteristics  of 
one  or  the  other  ere  known  or  can  ba  surmised. 

2.  Some  means  of  eliminating  the  effects  of  measurement  system  noise  from 
probability  density  functions,  power  spectra,  ate  of  the  measured  data. 

3.  A  mean*  of  measuring  or  inferring  the  physical  shape  of  the  image  of  the 
disturbances  on  the  detector, 

4.  A  theoretical  treatment  of  tha  problem  to  allow  overall  guidance  of  the 
experimental  effort. 
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Summary  of  Need* 


Thus  a  summary  of  our  nssds  to  oomplsts  the  design  of  a  auooessful 
experiment  inoludest 

1)  Some  means  of  "unconvolving11  two  effects  when  some  character¬ 
istics  of  one  or  the  other  are  known  or  oan  be  surmised, 

2)  Some  means  of  eliminating  the  effeots  of  measurement  system 
noise  from  probability  density  functions,  power  spectra,  etc  of  the 
measured  data. 

3)  A  means  of  measuring  or  inferring  the  physical  shape  of  the 
image  of  the  disturbances  on  the  deteotor. 

li)  A  theoretical  treatment  of  the  problem  to  allow  overall 
guidance  of  the  experimental  effort, 


THE  ANALYSIS  OF  A  SUCCESS-FAILURE  TIME  SERIES  WITH  AN 
APPRECIABLE  NUMBER  OF  MISSING  OBSERVATIONS 


Robert  P.  Lee 

Atmospheric  Sciences  Leboretory 
White  Sends  Missile  Range,  New  Mexico 

ABSTRACT,  The  calculetion  of  the  power  spectrum  of  e  time  caries 
whose  elements  were  either  "success"  or  "failure"  and  where  an  appreciable 
number  of  the  elements  were  missing  is  reported. 

INTRODUCTION.  This  paper  is  the  result  of  a  request  for  a  power 
spectrum  analysis  of  a  time  series  derived  from  surface  meteorological 
data  which  had  been  collected  hourly  over  about  19  years  at  White  Sands 
Missile  Range.  Briefly,  if  certain  wind,  relative  humidity,  and  time 
of  day  conditions  were  met  and  the  visibility  was  less  than  a  certain 
number  of  miles,  the  day  was  to  be  considered  a  success.  A  day  which 
was  not  a  success  was  labelled  a  failure  unless  no  data  was  available 
for  that  day.  Of  the  6909  elements  of  the  time  series,  1641  were 
successes,  4784  were  failures,  and  484  Were  missing. 

An  examination  of  the  data  revealed  that  the  missing  elements  were 
in  general  quite  random,  except  for  about  four  years  near  the  middle 
of  the  time  span  during  which  very  little  Sunday  data  had  been  recorded. 
The  Sunday  data  that  was  recorded  showed  what  appeared  to  be  a  sig¬ 
nificantly  higher  percentage  of  pluses,  indicating  that  the  Sunday 
data  during  this  period  might  not  match  the  rest  of  the  time  series. 

PROCEDURE.  A  plus  one  was  assigned  to  a  success,  a  minus  one  to  a 
failure,  and  a  zero  was  used  as  a  key  to  the  missing  elements.  Subtracting 
the  mean  from  each  of  the  data  points  and  dividing  by  the  standard 
deviation  scaled  the  time  series  to  unity  variance  with  zexo  mean. 

The  series  now  contained  1641  elements  with  a  value  of  1.71  and 

4784  elements  with  a  value  of  -.59,  and  as  before  484  elements  misBing. 

The  first  197  autocovariance  coefficients  (1)  were  calculated  from 
the  equation 


^  t  **  XJ+k  ’  k"°'  lf  2 . 196 

where  was  the  number  of  pair  products  present  at  lap,  k.  (The  value  of 

196  for  the  maximum  lag  waa  chosen,  after  several  trials,  because  the 
exact  reciprocal  of  7,  14,  and  28  days  would  be  among  the  resulting 
frequencies.)  The  finite  cosine  transforms  of  the  autocovariances  were 
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smoothed  using  a  three  point  filter  (spectral  window)  having  the  weights 
.25,  .50,  .25  producing  the  power  spectrum  shown  in  Figure  1.  There  ere 
two  very  obvious  peaks  at  frequencies  corresponding  to  periods  of 
(1)  7  days  and  (2)  3  V2  days.  The  one  labelled  (3),  almost  indistin¬ 
guishable  in  the  noise,  corresponds  to  a  period  of  2  V3  days  and  is 
in  harmonic  relationship  with  the  first  two.  Since  the  time  series  had 
unity  variance,  the  sum  of  the  power  at  all  frequencies  was  unity, 
and  the  power  axis  is  so  scaled.  About  18X  of  the  total  power  ie  in 
the  four  lowest  bands. 

A  suggestion  was  made  that  the  missing  data  be  filled  in  with  pluses 
and  minuses  drawn  randomly  from  a  population  having  the  same  proportion 
as  the  original  data.  Since  the  background  noise  was  already  high,  it  was 
assumed  that  the  additional  noise  introduced  by  this  procedure  might  not 
be  noticeable.  The  resulting  spectrum  is  shown  in  Figure  2.  Exactly 
the  same  frequencies  are  still  the  predominant  ones.  In  fact,  the 
spectrum  ie  almost  unchanged. 

As  a  final  test  of  the  effect  of  the  missing  data  and  to  eliminate 
any  possibility  that  bad  data  taken  on  Sunday  had  influenced  the  results, 
all  the  Sunday  data  points  for  the  entire  19  years  were  changed  to 
blanks.  The  resulting  spectrum  is  given  in  Figure  3.  Peak  (1)  has  been 
sharpened  some,  peak  (2)  la  almost  unchanged,  and  peak  (3)  is  now  in  the 
noise. 

DISCUSSION.  It  is  obvious,  once  it  is  pointed  out,  that  it  is  possible 
to  wipe  out  points  arbitrarily  from  a  time  series  and  still,  using  the 
cosine  transform  of  the  autocovar lances,  obtain  almost  exactly  the  same 
power  apectrum.  If  the  missing  points  are  representative  of  the  entire 
set,  the  autocovariances,  and  therefore  the  power  spectrum,  will  be 
unchanged.  In  this  case,  removing  the  questionable  data  served  to 
raise  the  peak  corresponding  to  a  period  of  7  days. 

Two  very  valuable  papers  on  the  material  covered  in  this  paper  have 
been  written  by  Dr,  Richard  H,  Jones,  now  with  the  Department  of  Information 
and  Computer  Sciences,  University  of  Hawaii  (2,3).  They  contain  detailed 
mathematical  derivations  of  the  effect  of  missing  observations  on  the 
variance  of  the  spectral  estimates,  approximate  degrees  of  freedom,  etc. 

It  is  to  be  expected  that  a  large  percentage  of  the  energy  present 
would  be  in  the  very  low  frequencies  corresponding  to  periods  of  one 
year  and  greater,  thus  complicating  the  analysis  at  higher  frequencies. 

CONCLUSION.  The  power  spectrum  calculations  show  a  predominant 
amount  of  power  at  frequencies  of  one  cycle  per  year  and  lover.  From 
meteorological  experience,  periods  of  3  1 / 2  and  7  days  are  known  to  be 
very  reasonable.  The  results  of  this  paper  show  that  a  time  series 
whose  elements  take  only  the  two  values  "success"  or  "failure"  is  no  bar 


to  the  calculation  of  a  reasonable  power  spectrua,  even  though  an  appreciable 
nuabar  of  the  observations  are  aisaing. 
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FIGURE  2  -  POTER  SPECTRUM  IF  ORTA  WITH  HISSING  0BSERVRT1SNS  RflNWHLY  FILLED 


FIGURE  3  -  PMER  SPECTRUM  WF  ORTA  fITH  ALL  SUNDRY  ORTR  ELIMINATED 


